Using GetPixel to identify objects inside a game window
This Tutorial is meant to be beginner friendly, provide information while not being dry as a bone
You are probably familiar with the concept of reading pixel color at a position on the screen. This tutorial goes one step further and looks for an "object" of unknown location which can not be identified by a single pixel color.
I also uploaded a video yesterday which includes this example:
feedback would be very much appreciated (auch gerne auf Deutsch)
Lets get the basics out of the way:
The following funciton provides pixel color at position x and y.
Code:
COLORREF GetPixel(HDC hdc,int x,int y);
Since we want to look for something inside a game window we first get a handle to the window and then the desired device context like so:
Code:
HWND hWND = FindWindow(NULL, L"Zuma Deluxe 1.1.0.0"); HDC hDC = GetDC(hWND); //handle to the device context //Do stuff with hDC ReleaseDC(hWND,hDC); //make sure to release after use!
Code:
RECT rect = {};
GetClientRect(hWND, &rect);
int right_border = rect.right;
int bottom_border = rect.bottom;
Now to a practical example:
As my target I picked the game Zuma. What I wanna know is the location of the frog. Starring my enemy right into the eye I realized that those will probably be decent pixels to search for. Turns out that only within those dead eyes there are 2 pixels next to each other having a specific color code:
After having found the first eye we already kinda know the frogs position but since the frog can turn 360° I wanted to get a good estimate of the center of the frog. So the idea is to look for the second eye as well then add those two positions up and divide by two to get the center position between those two eyes. The search area for the second eye is restricted:
Since we started top left going down and then right we only need to search to the right of the first eye as well as not too close and not too far away:
Equipped with the basics as well as plan for action all that is left to do is write it all down:
Code:
HWND hWND = FindWindow(NULL, L"Zuma Deluxe 1.1.0.0");
std::cout << "looking for frog start" << std::endl;
HDC hDC = GetDC(hWND);
COLORREF color;
COLORREF characteristic_color = 15398653; // Color at the Shiny Spot of the eye
int first_eye_x = 0;
int first_eye_y = 0;
RECT rect = {};
GetClientRect(hWND, &rect); // Stores info about the game window size
for (int i = 0; i < rect.right; i++) { // loop through x coordinates starting at 0
for (int j = 0; j < rect.bottom; j++) { // loop through y coordinates starting at 0
color = GetPixel(hDC, i, j);
if (color == characteristic_color) { // ist this an eye?!
color = GetPixel(hDC, i, j + 1);
if (color == characteristic_color) { // same color in the next pixel!! we got an eye
first_eye_x = i;
first_eye_y = j;
std::cout << "first eye: x: " << first_eye_x << " y:" << first_eye_y << std::endl;
for (int k = first_eye_x - 35; k < first_eye_x + 35; k++) { // search around first eye for second eye
for (int l = first_eye_y - 35; l < first_eye_y + 35; l++) {
color = GetPixel(hDC, k, l);
if (abs(k - first_eye_x) > 10 && abs(l - first_eye_y) > 10 && color == characteristic_color) { // found the second eye!!
std::cout << "second eye: x: " << k << " y:" << l << std::endl;
std::cout << "frog center at: x: " << (abs(first_eye_x + k) / 2) << " | y:" << (abs(first_eye_y + l) / 2) << std::endl;
return 0;
}
}
}
}
}
}
}
ReleaseDC(hWND, hDC);
This does work BUT I want to give a bit of a warning: GetPixel is a very slow function. So if you need information fast and/or your search area is big this method is probably not a good choice!
