There's this handy script which will let you install and setup OpenVPN on your server the easiest way possible:

After it's finished, you will get a *.ovpn file, which is your OpenVPN client profile. Just copy that file to your PC at location A and load it up in OpenVPN Client (

). This will let you access the Internet through your VPS, but you said you also need some ports open. You can read up on how to accomplish that here: