The point I was trying to make clear is that, even if you use bytes, you are not saving any space - the compiler will treat them as ints for most things. (With the exception of array of bytes, or collections of bytes in structs). When you have a single byte, it will occupy 4 bytes because of the way the compiler aligns data to memory words.
And note there that I'm talking about storage (such as fields in a class). When talking about local variables, they are almost entirely treated as ints for their short lifetime. Size is really not an issue here, since the memory is only occupied for the duration of the method.
I'll give an example to make it more clear.
Code:
byte x = 255;
byte y = 1;
byte z = (byte)(x + y);
The CIL for this would be something like
Code:
ldc.i4 255
stloc.0
ldc.i4 1
stloc.1
ldloc.0
ldloc.1
add
conv.u1
stloc.2
The CLR stack is a fixed width of ints though, so all those ldloc and stloc are moving ints around in memory - not bytes. We can see this by dissassembling it and snipping out the relevant parts.
Code:
mov dword:[ebp-4],0
mov dword:[ebp-8],0
mov dword:[ebp-12],0
mov dword:[ebp-4], 255
mov dword:[ebp-8], 1
mov eax, dword:[ebp-4]
add eax, dword:[ebp-8]
and eax, 0xff
mov dword:[ebp-12], eax
All instructions are using dwords (int). The ASM isn't optimised to use smaller register sizes like AL, as you might've expected. Instead, the conv.u1 instruction is simply converted to (& 0xFF), removing the overfill of a byte.
If you convert the original C# to use int instead of byte - the difference will be the conv.u1 instruction is gone, and the matching and eax, 0xff CPU instruction with it. There's actually no other difference: they are treated equally as ints, and using bytes is doing nothing but adding extra instructions.
Of course, there are meaningful times when it's desirable to use bytes, if you are specifying the intention that something should be a byte. But if you are performing arithmetic on a value, and you want to use bytes purely for the sake of "saving space", you're wasting instructions.