How do you write efficient sockets?

[Xio.]

Originally Posted by Spirited

Stop it. This is ridiculous. Allocating memory for a buffer isn't expensive, and the socket system already allocates a buffer for you. You know what is expensive? Locking. Concurrency doesn't mean something can run in parallel. Having a "ConcurrentStack" popping off buffers means it's going to lock up there each time you want to get a buffer, which is far less efficient than just allocating stack memory

Allocations are cheap. garbage collections are not. According to profiling (ANTS) the sockets were the biggest cause of allocations, with 4 bots hunting the time spent on garbage collection was 3.6% of the total CPU time. The bufferpool now shows that the time spent running GC's is less than 1% of the total cpu time.

I agree on the locking part although concurrent collections are very well optimized they are slower than what you suggested. I don't see a bottleneck there though.

I had about 1 Gen0 GC every 2.5-3 seconds without the buffer pool, that's basically zero now.

[Spirited]

Originally Posted by Xio.

Allocations are cheap. garbage collections are not. According to profiling (ANTS) the sockets were the biggest cause of allocations, with 4 bots hunting the time spent on garbage collection was 3.6% of the total CPU time. The bufferpool now shows that the time spent running GC's is less than 1% of the total cpu time.

I agree on the locking part although concurrent collections are very well optimized they are slower than what you suggested. I don't see a bottleneck there though.

I had about 1 Gen0 GC every 2.5-3 seconds without the buffer pool, that's basically zero now.

It is as I said. If there's additional garbage collection happening as a result of how you're doing sockets, then you're doing them incorrectly. As I said, sockets have an allocated buffer ready for you to use in C#. Unless you're doing things like copying that buffer around a few times, there's no reason for it to be having allocation problems.

[Xio.]

Originally Posted by Spirited

It is as I said. If there's additional garbage collection happening as a result of how you're doing sockets, then you're doing them incorrectly. As I said, sockets have an allocated buffer ready for you to use in C#. Unless you're doing things like copying that buffer around a few times, there's no reason for it to be having allocation problems.

Show me how you create packets to be sent please. Show me that you don't allocate a new byte array for the packet you're going to send.

The whole point of the pool is to create packet from it, recycle them and re-use existing buffers. Of course there is a buffer you send/receive but usually you copy it once you receive it so you can call beginreceive again without overwriting the previous buffer. That copy operation is an allocation.

[Spirited]

Originally Posted by Xio.

Show me how you create packets to be sent please. Show me that you don't allocate a new byte array for the packet you're going to send.

Oh, you're talking about sending packets. Yes, you will be allocating memory, but I still strongly argue that time in allocating an array is much smaller than the time it takes to synchronize and lock. Allocating is just a stack operation. Also, the lifetime of these packets shouldn't be large - it's supposed to be just within the parent function creating it. I don't see where you're getting these weird statistics from. I'd actually go as far to call bullshit.

[Best Coder 2014]

Originally Posted by Spirited

Oh, you're talking about sending packets. Yes, you will be allocating memory, but I still strongly argue that time in allocating an array is much smaller than the time it takes to synchronize and lock. Allocating is just a stack operation. Also, the lifetime of these packets shouldn't be large - it's supposed to be just within the parent function creating it. I don't see where you're getting these weird statistics from. I'd actually go as far to call bullshit.

This is C# we're talking where arrays are heap allocated and garbage collected (hint: allocation on the heap is not "just a stack operation").
Allocation is still cheap and fast, though, but garbage collection pauses is obviously what Yuki is worried about here, which is pretty clear if you actually read and understand his posts.

[Xio.]

Originally Posted by Spirited

Oh, you're talking about sending packets. Yes, you will be allocating memory, but I still strongly argue that time in allocating an array is much smaller than the time it takes to synchronize and lock. Allocating is just a stack operation. Also, the lifetime of these packets shouldn't be large - it's supposed to be just within the parent function creating it. I don't see where you're getting these weird statistics from. I'd actually go as far to call bullshit.

Not only about sending packets but yes. As I said, allocations are cheap, GC isnt.

I'm using the buffer to create incoming packets from them without a single extra allocation and I'm doing the same for packets I want to send. I take out a buffer object, write the data into it, pass it directly to the crypto to work on it and then to the socket to send it. I'm not copying packets, I'm not creating a new array, I re-use what I allocated in advance and then recycle it once it has been sent or processed.

This is C# we're talking where arrays are heap allocated and garbage collected (hint: allocation on the heap is not "just a stack operation").
Allocation is still cheap and fast, though, but garbage collection pauses is obviously what Yuki is worried about here, which is pretty clear if you actually read and understand his posts.

Thank you for reading and understanding. I thought I wasn't able to get my point across.

[Spirited]

Why are you worried about garbage collection pauses? Is sounds like a lot of misdirection. Are you sure it's not a case of mishandling the data once it's allocated? C#'s garbage collection shouldn't be a problem. In addition, if allocation isn't the problem, then how is this factory class you cooked up solving anything?

[Xio.]

Originally Posted by Spirited

Why are you worried about garbage collection pauses?

Because I'm trying to make a conquer server run on a Raspberry Pi Zero. It has 512mb of ram, a single core ARM CPU that's slow as fuck. GC is blocking for 21% of the total CPU time.

Is sounds like a lot of misdirection. Are you sure it's not a case of mishandling the data once it's allocated?

Yes I'm sure about that. I'm casting my data to or from a struct using the code I posted. It's as fast as it can get according to a bunch of people on CodeReview.

C#'s garbage collection shouldn't be a problem.

Then again, IT IS.

In addition, if allocation isn't the problem, then how is this factory class you cooked up solving anything?

Because that "factory" class I "cooked up" keeps a number of byte arrays pooled which will be used, recycled and used again.

If you allocate a new byte array the GC will have to collect it eventually - sooner or later. You can't explicitly destroy objects in C#. With my cooked up class I can completely avoid creating new byte arrays for packets.

Less Allocations -> Less time spent on GC.

Oh and before you come back at me with the obvious:

A conquer server on a raspberry pi zero is a waste of time, will never work and you are an idiot.

I'm using conquer servers to apply things I learn since they are usually the perfect testbench. I don't intend to run a public server, I just like to work on slow hardware to figure out ways to improve my code. It's just something I do for fun and some learning.


Also, imho this kind of optimization makes a lot of sense. Why allocate new shit all the fkin time when you can avoid it? It's so simple to pull off and makes a measurable difference on low end hardware. Heck even on a slow VPS I tested it on I saw improvements.

[CptSky]

Originally Posted by Xio.

Yes I'm sure about that. I'm casting my data to or from a struct using the code I posted. It's as fast as it can get according to a bunch of people on CodeReview.

So MsgAction is a structure? If so, every time you pass it you creates a copy (e.g. calling the implicit cast operator? Creating a copy...).

Also, you should probably change the size fo the buffers (850) to something more computer friendly to avoid internal fragmentation of the heap.


P.S. Fang, memory allocation is cheap in native languages. It isn't the same in languages with GC. Creating too many objects with a short life-span will put a lot of pressure on the GC, at that point, you're better to recycle your memory instead.

[Xio.]

Originally Posted by CptSky

So MsgAction is a structure? If so, every time you pass it you creates a copy (e.g. calling the implicit cast operator? Creating a copy...).

Also, you should probably change the size fo the buffers (850) to something more computer friendly to avoid internal fragmentation of the heap.


P.S. Fang, memory allocation is cheap in native languages. It isn't the same in languages with GC. Creating too many objects with a short life-span will put a lot of pressure on the GC, at that point, you're better to recycle your memory instead.

Oh ***. I didn't realize that applies to operators... Thanks a ton!

[CptSky]

Originally Posted by Xio.

Oh ***. I didn't realize that applies to operators... Thanks a ton!

Well that would need some verification, but an operator is basically a function call so I would think so. Anyway, the point is mostly that you should make sure that everywhere in your code you pass the messages by references (ref keyword) as they will be copied otherwise. Maybe encapsulating the structure in a class (e.g. Info encapsulated in MsgAction) could be safer.

[Spirited]

See, that's what I needed to know. I have no idea how garbage collection is going to run on a Raspberry Pi. Probably horribly **** as you've benchmarked it, though I'd expect much better. Why you're doing this on a Raspberry Pi Zero is a very good question... but you're already aware of that, so I won't pester further. Have fun doing whatever this is.

[KraHen]

To be fair on the Pi I'd just lose any managed language and go closer to the hardware (creating wonderfully new problems instead but getting more control), but that's just me.

[Xio.]

Originally Posted by CptSky

Well that would need some verification, but an operator is basically a function call so I would think so. Anyway, the point is mostly that you should make sure that everywhere in your code you pass the messages by references (ref keyword) as they will be copied otherwise. Maybe encapsulating the structure in a class (e.g. Info encapsulated in MsgAction) could be safer.

I couldn't find any information about if the implicit operators copy value types but I wrote a quick test (1kk creating a new struct, casting to byte[] and restoring the struct from the byte[]) and It takes roughly 85ms to complete. Without implicit casting it drops to 72ms but then again, that difference could basically be the method call itself. I'll double check with ANTS later today, I think I saw something about value type copy counts in there.

I also tried using stackalloc and passing pointers, that resulted in 49ms on average.. Could be worth considering. I just don't know how to free those objects, the GC doesn't act on arrays allocated using stackalloc afaik.

This is exciting :3

[CptSky]

Originally Posted by Xio.

I couldn't find any information about if the implicit operators copy value types but I wrote a quick test (1kk creating a new struct, casting to byte[] and restoring the struct from the byte[]) and It takes roughly 85ms to complete. Without implicit casting it drops to 72ms but then again, that difference could basically be the method call itself. I'll double check with ANTS later today, I think I saw something about value type copy counts in there.

I also tried using stackalloc and passing pointers, that resulted in 49ms on average.. Could be worth considering. I just don't know how to free those objects, the GC doesn't act on arrays allocated using stackalloc afaik.

This is exciting :3

You don't seem to understand how stackalloc works. And as such, I highly recommend you to avoid it. It's not as easy as you think to properly use and I've done some big **** with it in the past.

If you really want to go to the pointers route. You should alloc one big buffer using AllocHGlobal, and than, when requested a buffer, return a subblock of it (using pointer arithmetics). This pointer can be casted to a struct pointer as you won't re-allocate any memory to handle the packet structure. I used something similar (but without a memory pool) in COPS v6 (the original version), but found that the pointer syntax in C# was too limiting.