[TUTORIAL] Reversing the graphics pipeline

[Cyrex']

Hey guys, this tutorial will show you basic static reverse engineering or analysis of a dump. In this case I will show you something of S4's graphics pipeline. The practical example will be the D3D Font. This tutorial aims at a bit advanced members and requires asm knowledge. At the end of the tutorial, I will also give you a tip on how to aquire a static pointer.


D3D Font

 Spoiler

Here, we found an interesting string that will help us to get a starting point.
You could also instead search for D3D API calls but since the IAT of my dump is destroyed and I don't want to fix it, I can't do that.



We are going to trace the string back to a code reference, to see where it
is used, and how.

Now we need to use logic - there is a conditional jump before our error,
which is the only one that will jump about our error message. Soo...
The only possible solution is ebp+var_44. ebp, which everyone should be clear, is btw. the stack ptr which is mostly used in functions instead of esp.

If var_44 is equal to zero, then a specific flag in the cpu will be set. If not it will give us a value that will indicate that it is bigger or smaller. 'jz' only checks the equality flag. Assuming it's equal it will jump, we now know that var_44 must be a variable that can say something about our state of the D3D font.




Tracing var_44 back, we can realize that it is set from eax, which is also set in the call before.



This call brings us to a jump which will jump to a probably DirectX DLL in which the function gets called. Since I didn't rebuild the IAT(hence, it's destroyed) it's not displayed as D3DXCreateFont(probably).


What does that mean to us?
To validate it's correct, you could hook this function and manipulate the value of the local var and see what S4 does. With this information you can simply grab the font ptr.

It's the same play with all other modules of the D3D pipeline.
Just find an error string inside of the exe and trace it back.
You can also try to find cross references and see if you can find a static ptr.
That'd be the best case.

How to get device?

 Spoiler

.text:00E9543A                 mov     edx, [ebp+var_40]
.text:00E9543D                 push    edx
.text:00E9543E                 mov     eax, [ebp+var_2C]
.text:00E95441                 mov     ecx, [eax+694h]
.text:00E95447                 mov     edx, [ebp+var_2C]
.text:00E9544A                 mov     eax, [edx+694h]
.text:00E95450                 mov     ecx, [ecx]
.text:00E95452                 push    eax
.text:00E95453                 mov     edx, [ecx+40h]
.text:00E95456                 call    edx
.text:00E95458                 mov     [ebp+var_34], eax
.text:00E9545B
.text:00E9545B loc_E9545B:                             ; CODE XREF: sub_E95200+21Aj
.text:00E9545B                 cmp     [ebp+var_34], 0
.text:00E9545F                 jz      short loc_E9547F
.text:00E95461                 mov     eax, [ebp+var_34]
.text:00E95464                 push    eax
.text:00E95465                 push    offset aErrorCrender_1 ; "[ ERROR ]  CRenderer_D3D::Init  [ Creat"...
.text:00E9546A                 call    sub_1FD8D0
.text:00E9546F                 push    eax
.text:00E95470                 call    sub_D55AD0
.text:00E95475                 add     esp, 0Ch
.text:00E95478                 xor     al, al
.text:00E9547A                 jmp     loc_E95BDB

var_2C = this pointer
eax + 0x694 = probably a class pointer to a render class

mov     ecx, [ecx]

get vtable.

add eax to parameter stack list(push it), which is the instance or class pointer of the render class.
call the 16th function of the vtable. move the result in var_34.

you nubs may ask what to do now?
get the crappy render class, get its vtable and emulate the vtable call

template< typename Function > Function EmulateVirtual(void* _VMT, int Index)
{
	void*** _TVMT = (void***)_VMT;

	void** VMT = *_TVMT;

	void* _Address = VMT[Index];

	return (Function)(_Address);
}

usage:

return EmulateVirtual< _FunctionType >(this, 16)(this, params);

where _FunctionType is a typedef of the function, whose calling conventions is ALWAYS __thiscall on x86

another thing you can do is reverse the named function + class to get more and more stuff.
also in the function it will probably grab the device ptr from somewhere.

credits: other forum

Hope you liked the tut.

[Shin..]

Can u please show some ingame-screenshots??

[Cyrex']

Originally Posted by ScreamStream

Can u please show some screen shots??

There are screenshots
EDIT: this is a tutorial and not a preview...

[RingleRangleRob]

Gj

Originally Posted by ScreamStream

Can u please show some ingame-screenshots??

do it yourself its an tutorial.

[Aquaforce]

Useful