You are struck with two very strong disadvantages here.
Binary server, and obviously, the client's also in binary form.
Now if you did what korvacs was saying, pipe them threw some sort of proxy before the data is sent to your server, that would work, but what's to stop someone from just bypassing the proxy and sending data directly to your server instead?
If you had a source server, C# or whatever it may be, you could forward all data threw to a proxy, modify/ re-encrypt or do whatever you like to the packets, and then have all the data processed threw the server.
This would be a way to completely stop aimbotting (at least until someone wants to write a proxy for your proxy and then send it to the game server), speedhacking and what have you.
But using a binary server, you really have no options as far as protection goes.
An alternative to someone with programming experience, write a driver, keep the service running 24/7 on the machine and have it monitor the file "X:\MyServer\MyServer.exe". If that file is renamed (or just goes out of site to the scanner) then create a new one with a backup stored in a hidden folder on the computer, and delete any executable files inside that folder which you don't define.
It's really flawed though, just because windows doesn't offer any support for preventing file renaming, moving, deleting etc etc (Wonder why? -> Hackers maybe?)
Anyways, you're options are too limited and for you, I'm sorry to say, it can't be done.
Case closed?