D3D hook failed on win8

xoraxax · 09/05/2014, 21:27

Hello.
I am trying to hook endScene this way:

Code:

#include <d3d9.h>
#include <d3dx9.h>
 
#pragma comment(lib, "d3d9.lib")
#pragma comment(lib, "d3dx9.lib")
 
 
void *DetourFunc(BYTE *src, const BYTE *dst, const int len) 
{
BYTE *jmp = (BYTE*)malloc(len+5);
DWORD dwback;
VirtualProtect(src, len, PAGE_READWRITE, &dwback);
memcpy(jmp, src, len); jmp += len;
jmp[0] = 0xE9;
*(DWORD*)(jmp+1) = (DWORD)(src+len - jmp) - 5;
src[0] = 0xE9;
*(DWORD*)(src+1) = (DWORD)(dst - src) - 5;
VirtualProtect(src, len, dwback, &dwback);
VirtualProtect(jmp,len+5,PAGE_EXECUTE_READWRITE,&dwback);
return (jmp-len);
}
 
typedef HRESULT(__stdcall* EndScene_)(LPDIRECT3DDEVICE9);
EndScene_ pEndScene;


HRESULT __stdcall EndScene(LPDIRECT3DDEVICE9 pDevice)
{
     return pEndScene(pDevice);
}
 
DWORD FindDevice(DWORD Len)
{
     DWORD dwObjBase = 0;
         
     dwObjBase = (DWORD)LoadLibrary(L"D3D9.DLL");
     while (dwObjBase++ < dwObjBase + Len)
     {
         if ( (*(WORD*)(dwObjBase + 0x00)) == 0x06C7
           && (*(WORD*)(dwObjBase + 0x06)) == 0x8689
           && (*(WORD*)(dwObjBase + 0x0C)) == 0x8689
           ) { dwObjBase += 2; break; }
     }
     return( dwObjBase );
}
 

void Hook()
{
     PDWORD VTable;
     *(DWORD*)&VTable = *(DWORD*)FindDevice(0x128000);
pEndScene = ( EndScene_ )DetourFunc((PBYTE) VTable[42],(PBYTE)EndScene, 5);
 
Sleep( 100 ); 
}
 

int WINAPI DllMain(HINSTANCE hInst,DWORD reason,LPVOID reserved)
{
switch(reason)
{
case DLL_PROCESS_ATTACH:
     CreateThread(0, 0, (LPTHREAD_START_ROUTINE) Hook, 0, 0, 0)
break;
}
return true;
}

It works great on win7 but crashes in win8. Does anyone have an idea why does it happen and how to fix it?

Mostey · 09/06/2014, 16:55

#moved

Terrat · 09/06/2014, 17:36

Quote:

Originally Posted by xoraxax

Hello.
I am trying to hook endScene this way:

Code:

#include <d3d9.h>
#include <d3dx9.h>
 
#pragma comment(lib, "d3d9.lib")
#pragma comment(lib, "d3dx9.lib")
 
 
void *DetourFunc(BYTE *src, const BYTE *dst, const int len) 
{
BYTE *jmp = (BYTE*)malloc(len+5);
DWORD dwback;
VirtualProtect(src, len, PAGE_READWRITE, &dwback);
memcpy(jmp, src, len); jmp += len;
jmp[0] = 0xE9;
*(DWORD*)(jmp+1) = (DWORD)(src+len - jmp) - 5;
src[0] = 0xE9;
*(DWORD*)(src+1) = (DWORD)(dst - src) - 5;
VirtualProtect(src, len, dwback, &dwback);
VirtualProtect(jmp,len+5,PAGE_EXECUTE_READWRITE,&dwback);
return (jmp-len);
}
 
typedef HRESULT(__stdcall* EndScene_)(LPDIRECT3DDEVICE9);
EndScene_ pEndScene;


HRESULT __stdcall EndScene(LPDIRECT3DDEVICE9 pDevice)
{
     return pEndScene(pDevice);
}
 
DWORD FindDevice(DWORD Len)
{
     DWORD dwObjBase = 0;
         
     dwObjBase = (DWORD)LoadLibrary(L"D3D9.DLL");
     while (dwObjBase++ < dwObjBase + Len)
     {
         if ( (*(WORD*)(dwObjBase + 0x00)) == 0x06C7
           && (*(WORD*)(dwObjBase + 0x06)) == 0x8689
           && (*(WORD*)(dwObjBase + 0x0C)) == 0x8689
           ) { dwObjBase += 2; break; }
     }
     return( dwObjBase );
}
 

void Hook()
{
     PDWORD VTable;
     *(DWORD*)&VTable = *(DWORD*)FindDevice(0x128000);
pEndScene = ( EndScene_ )DetourFunc((PBYTE) VTable[42],(PBYTE)EndScene, 5);
 
Sleep( 100 ); 
}
 

int WINAPI DllMain(HINSTANCE hInst,DWORD reason,LPVOID reserved)
{
switch(reason)
{
case DLL_PROCESS_ATTACH:
     CreateThread(0, 0, (LPTHREAD_START_ROUTINE) Hook, 0, 0, 0)
break;
}
return true;
}

It works great on win7 but crashes in win8. Does anyone have an idea why does it happen and how to fix it?

Why did you want to know if this work on win 7 i cant see any thing what seems like a box

Mostey · 09/06/2014, 18:15

Quote:

Originally Posted by Dreamsläps

Why did you want to know if this work on win 7 i cant see any thing what seems like a box

I can't see where he stated that he is drawing anything.

@xoraxax
Which OS architecture do you use? You might want to change your DWORD datatype to something platform independent that guarantees the size of a pointer. This is important since pointers got different sizes on different architectures. On x86, a pointer is usually 4 bytes long while a pointer on x64 is 8 bytes long. You should be aware of that when addressing memory.

uintptr_t may be a good solution.

Terrat · 09/06/2014, 22:23

Quote:

Originally Posted by Mostey

I can't see where he stated that he is drawing anything.

@xoraxax
Which OS architecture do you use? You might want to change your DWORD datatype to something platform independent that guarantees the size of a pointer. This is important since pointers got different sizes on different architectures. On x86, a pointer is usually 4 bytes long while a pointer on x64 is 8 bytes long. You should be aware of that when addressing memory.

uintptr_t may be a good solution.

That is right but if he dont draw anything why the hack he know that this works

Mostey · 09/07/2014, 01:29

Quote:

Originally Posted by Dreamsläps

That is right but if he dont draw anything why the hack he know that this works

You know that he is hooking the EndScene function, do you? Meaning, he's writing the address of his own function into memory so everytime the game (or whatever he's addressing) calls the EndScene function, it get's redirected to the hooked one. Since he's backing up the address of the original function before writing, he may call the original one when returning.

Drawing things is optional, he doesn't have to draw anything. And it's crashing, because the hook can't be placed. (due to the size of the pointers)

xoraxax · 09/07/2014, 15:12

Well, it does work on win 7 x64/x32 (so pointers don't seem to fail). I can draw something - all drawing is omitted in that example. But exactly same code crashes on win 8. And i can not understand why. Is there any significant difference between win7 and win8 which i do not know?

phize · 09/07/2014, 15:35

You are overwriting only 5 bytes. Check d3d9.dll.

xoraxax · 09/07/2014, 15:46

Are d3d9.dll in win7 and d3d9.dll in win8 different?

ƬheGame · 09/08/2014, 14:45

Quote:

Originally Posted by xoraxax

Are d3d9.dll in win7 and d3d9.dll in win8 different?

BTW: Please start to get used to a betther coding style wit some comments and a style that is the same over the whole document. Its really hard to read your code.

Maybe this will help you, found it on google.

Code:

void Hook()
{
	MODULEINFO mInfo;

	DWORD dwD3D9 = NULL;

	OSVERSIONINFO WindowsVersion;
	WindowsVersion.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);
	GetVersionEx(&WindowsVersion);

	while(!dwD3D9)
	{
		dwD3D9 = (DWORD)GetModuleHandle("d3d9.dll");
	}

	GetModuleInformation(GetCurrentProcess(), (HMODULE)dwD3D9, &mInfo, sizeof(MODULEINFO));

	if(WindowsVersion.dwMajorVersion < 5 || WindowsVersion.dwMinorVersion >= 3)
	{
		MessageBoxA(NULL, "Your Operating System isn't supported!", "Attention!", NULL);
		exit(EXIT_SUCCESS);
	}	

	if(WindowsVersion.dwMajorVersion == 6 && WindowsVersion.dwMinorVersion == 2)//Windows 8
	{
		Main->D3D->Settings->dwEndScene = (int)mInfo.EntryPoint + 0xFFFFFFFFFFFDF736; 
		Main->D3D->Settings->dwReset = 0x72452A8F;
	}
	else if(WindowsVersion.dwMajorVersion == 6  && WindowsVersion.dwMinorVersion == 1)//Windows 7	
	{
		Main->D3D->Settings->dwEndScene = (int)mInfo.EntryPoint + 0x20D5A; 
		Main->D3D->Settings->dwReset = 0x6B9DF251;
	}
	else if(WindowsVersion.dwMajorVersion == 5  && WindowsVersion.dwMinorVersion == 1)//Windows XP
	{
		Main->D3D->Settings->dwEndScene = 0x4FE571B0; 
		Main->D3D->Settings->dwReset = 0x4FE136B0;
	}	

	pEndScene = (oEndScene)DetourFunction((PBYTE)Main->D3D->Settings->dwEndScene, (PBYTE)&EndScene);
	pReset   = (oReset)DetourFunction((PBYTE)Main->D3D->Settings->dwReset, (PBYTE)&Reset);	
}

xoraxax · 09/08/2014, 15:38

Quote:

Originally Posted by ƬheGame

Please start to get used to a betther coding style wit some comments and a style that is the same over the whole document. Its really hard to read your code.

Sorry for that - I am not a coder, so don't have good coding habits.

And ty all for your tips, i'll try to find out what happens again as soon as i get my hands on win8 pc.