Hello Ya'll, looking for some help which is semi-relevant to CO2, but if this need to go somewhere else, let me know.
If any of you played Monster & Me 2.5 back in the day, the file structure for early MAM and CO are pretty similar. I've been working on a project to revive MAM, and have used quite a bit of the resources and information on this forum to great effect (Thanks everyone!). Unfortunately, I've hit a stone wall for nearly 2 weeks now, and want to see if ya'll have any thoughts or help to provide.
TQ has a semi-custom file extension called RLE, which obviously stands for run-length encoding. It follows standard structure for the encoded image data with packet headers and 2 byte length pixels. That's where the easy part ends. The header of the file doesn't follow any known image header standards. I am able to parse the image data and output an image, but the pixel coloration is completely wrong because it looks to be color-mapped. Unfortunately the color map is some blob of bytes that I just can't figure out how to parse, and without a proper header, it's extremely hard to know how and what to read!
Have any of you come across these before? I can upload samples of the files and the things I've managed to figure out so far if you want to take a look (parts of the header such as w/h and image data offset, the image data decoding algorithm etc).
I guess it's certainly possible, but I am not sure how to confirm it yet. My current assumption is that these were just a custom compressed TGA file, since that is what all the other 'sprites' are in the game.
All the image data is split into three files: data, character, pet WDF. I reached out to Cloud Wu (developer of Windsoul++, original engine for MAM CO and other TQ games) and got the full copy with all source code (all the ones posted online are missing any number of pieces). Was able to use that to extract the WDF files, sans file names since the .lst seem to be missing.
Most of data is in TGA/BMP/JPG format. The exception is 99% of character and pet is in this nasty RLE format, which accounts for some 40,000 files!!! There were literally only 2 files in character that were not RLE, and those were in TGA format.
Since this is back from 2001, I don't know that they were using textures inasmuch as converting data to a 2D bitmap. Windsoul++ only seems to handle bitmaps and not as textures, though I'm not completely sure. Reading the source database and packet structure, they actually send sets of Hue/Sat/Bright as a packet to dictate alternate colors, instead of having multiple copies clientside. Not sure if this has any bearing, but I found it at least interesting
I attached a rar of some of the sample files and the results of the RLE decoding of the image data. As you can see, the colors are horribly wrong, so don't mind that It is of note, every RLE file starts with 'ND' for some reason. Maybe it means 'NetDragons', who knows. Usually littleEndian would flip the letters, so maybe its just a bad guess haha.
--------------
BTW, if you want the full Windsoul++ copy, just let me know. it's a free game engine anyways, though quite aged now, so I'd be glad to pass out the correct version Cloud Wu sent me.
The file header is of a different structure, so it doesn't recognize it as a WSprite. Furthermore, the color map data seems to be of variant length whereas the load() for WS seems to expect a finite 512 bytes of colormap data.
Do you have a screenshot of what is supposed to be the real colors? It can help to map them in the palette of the file and understand the format.
That, my good sir, is an excellent idea! It'll take a touch of hunting, to find an in game asset that matches one of the 30k monster images, but certainly doable. Now I at least have another avenue to start investigating without just staring at the hex editor for hours and trying random ideas.
Update: Seems like CptSky's suggestion was a no go. Pet colors in game are modified by the server provided hue/sat/bright with some sort of range/modifier. Raises the question on what colors these RLEs actually have. I can verify that there is 1 image for all 3 versions of each pet, and the above values are somehow applied to the RLE data. As for what the base image looks like, still a mystery!
Perhaps also of interest is the fact that the start of the image data encoding doesn't actually lineup with the number of pixels in the image. Drawing them from the end of the array results in a properly shaped images (with wrong colors), but that leaves a fair chunk of now decoded bytes following the same RLE syntax, but not used in the image. Some files don't have enough of these to cover the number of pixels, others have nearly 4x as many as the image pixels itself.
Starting to think this is a hopeless endeavor. I'd actually be tempted to pay someone to try and figure this out for me so I can focus my energies on other parts of this rebuild project. Surprised I got as far as I did with the image data. I've decoded 90% of the packets, and all the ciphers are functioning. This is the last step to writing a new game client that works with the server. What do ya'll think? Is it worth it? Not really familiar with the trading/black market aspect of the site though I've been trying to read the postings.
Starting to think this is a hopeless endeavor. I'd actually be tempted to pay someone to try and figure this out for me so I can focus my energies on other parts of this rebuild project. Surprised I got as far as I did with the image data. I've decoded 90% of the packets, and all the ciphers are functioning. This is the last step to writing a new game client that works with the server. What do ya'll think? Is it worth it? Not really familiar with the trading/black market aspect of the site though I've been trying to read the postings.
Is being able to open those files? Why not just reuse the existing client and have it connect to your server? Is there a server.dat file you can modify?
Is being able to open those files? Why not just reuse the existing client and have it connect to your server? Is there a server.dat file you can modify?
The project goal is to write an entirely new client as it hasn't been updated in many years (Since like 2001/2~). We have the original game server, account server, autopatch server, and database, and have been freely changing the database with our own tweaks. TQ actually provided the files to us themselves. Unfortunately, no source code and the client is buggy as hell and full of issues which really kills the experience.
We have all the clientside resources extracted from the WDF files, and ini files not packet into WDF. We have most of the packets decoded including the ciphers, cipher seed, and password cipher. The main stop now is that a lot of the game sprites are in these RLE files (Some 50k+ of them! yikes). All the rest are in TGA/BMP/JPG which we can read. Without a way to interpret the image data in the RLEs, we can't rewrite the client because all of the monsters, characters, and animated sprites in game all use this format.
We actually reached out to see if there was any of their prior programmers who could point us in the right direction. No followup as of yet =/
Can you post the code you have right now? And maybe upload more files to find correlation/do tests. Also, if you could find a specific RLE file, with the in-game color and the hue/sat/bri values.
I might take a look if I have some free time. I can't guarantee anything.
I have attached what I have so far as a RAR. Included my cpp code, a handful of RLE samples (all of the same monster), the current image output data I am getting for each, gifs of the actual monster with the proper colors (three variants, each color described in petColor.ini).
The same base RLE is used for each variant of the same pet, and different sets of colors are applied to three different zones of its image data. I assume this has something to do with the hue 32/96/128 values as well as the 8 range. This stays consistent for every single HSB values, though I am not yet clear on how it is implemented or what the original image looks like without the HSB changes.
The first 18 bytes of the file header has consistently followed this format:
bytes[2] "ND"
00
int width
int height
WORD (always 8)
WORD (always 1)
SHORT Image offset from end of file
The rest is all mystery data so far.
Here is my code thus far. Apologies for any lack of style or skill on the implementation >.<
Code:
#include <windows.h>
#include <iostream>
#include <fstream>
#include <sstream>
#include <iomanip>
#include <vector>
#include "FreeImagePlus.h"
using namespace std;
struct tColorRGBA
{
unsigned char r;
unsigned char g;
unsigned char b;
unsigned char a;
};
struct RLEheader {
BYTE id[2];
WORD u1;
DWORD width;
DWORD height;
WORD eight;
WORD one;
WORD endOffset;
WORD zero;
};
bool LoadRLE(string fileName);
tColorRGBA convertBytesToPixel(unsigned char* chunk);
bool write_truecolor_tga(string fileName, vector<tColorRGBA> data, unsigned width, unsigned height);
int main() {
string fileName = "turtle0";
LoadRLE(fileName);
return 1;
}
bool LoadRLE(string fileName) {
string rleFile = fileName + ".rle";
RLEheader header;
unsigned int bytesPerPixel;
//RLE parts
unsigned char packetHead;
unsigned char *chunk;
tColorRGBA pixel;
vector<tColorRGBA> pixels;
ifstream ifs(rleFile, ios::binary | ios::ate);
ifstream::pos_type pos = ifs.tellg();
int fileSize = pos;
ifs.seekg(0, ios::beg);
ifs.read((char*)&header, sizeof(RLEheader));
int imageOffset = fileSize - header.endOffset;
int dataSize = fileSize - imageOffset;
ifs.clear();
ifs.seekg(imageOffset);
BYTE *data = new BYTE[dataSize];
ifs.read((char*)data, dataSize);
bytesPerPixel = 2;
int bytesDone = 0;
//Parse Image Data as packet header followed by 2*count bytes
while (bytesDone < dataSize) {
chunk = new unsigned char[2];
packetHead = data[bytesDone];
//Run of the same pixel
if ((packetHead & 0x80)) {
packetHead &= 0x7F;
packetHead++;
chunk[0] = data[bytesDone + 1];
chunk[1] = data[bytesDone + 2];
pixel = convertBytesToPixel(chunk);
for (unsigned char b = 0; b < packetHead; b++) {
pixels.push_back(pixel);
}
bytesDone += bytesPerPixel;
}
//Sequence of different pixels
else {
packetHead &= 0x7F;
packetHead++;
for (unsigned char b = 0; b < packetHead; b++) {
chunk[0] = data[bytesDone + 1];
chunk[1] = data[bytesDone + 2];
pixel = convertBytesToPixel(chunk);
pixels.push_back(pixel);
bytesDone += bytesPerPixel;
}
}
bytesDone++;
}
ifs.close();
delete[] data;
data = NULL;
write_truecolor_tga(fileName, pixels, header.width, header.height);
pixels.clear();
return true;
}
tColorRGBA convertBytesToPixel(unsigned char* chunk) {
tColorRGBA result;
result.a = 255 * ((chunk[1] & 0x80) >> 7);
result.r = (chunk[1] >> 2) & 0x1F;
result.g = ((chunk[1] << 3) & 0x1C) | ((chunk[0] >> 5) & 0x07);
result.b = (chunk[0] & 0x1F);
// Convert 5-bit channels to 8-bit channels by left shifting by three
// and repeating the first three bits to cover the range [0,255] with
// even spacing. For example: http://www.ryanjuckett.com/programming/parsing-colors-in-a-tga-file/
// channel input bits: 4 3 2 1 0
// channel output bits: 4 3 2 1 0 4 3 2
result.r = (result.r << 3) | (result.r >> 2);
result.g = (result.g << 3) | (result.g >> 2);
result.b = (result.b << 3) | (result.b >> 2);
return result;
}
bool write_truecolor_tga(string fileName, vector<tColorRGBA> data, unsigned width, unsigned height)
{
int startPos = 0;
FIBITMAP *bitmap = FreeImage_Allocate(width, height, 32);
//Most of the images are 'flipped', but not all. method to identify direction not yet found
int row = height - 1;
for (unsigned y = 0; y < height; y++) {
BYTE *bits = FreeImage_GetScanLine(bitmap, row);
for (unsigned x = 0; x < width; x++) {
int pixelPos = ((y)* width) + x + startPos;
tColorRGBA pixel = data.at(pixelPos);
// Set pixel color
if (pixel.r == 0 && pixel.g == 0 && pixel.b == 0) bits[3] = 0;
else bits[3] = 255;
bits[2] = (char)pixel.r;
bits[1] = (char)pixel.g;
bits[0] = (char)pixel.b;
// jump to next pixel
bits += 4;
}
row--;
}
string tgaFile = fileName + ".tga";
FreeImage_Save(FIF_TARGA, bitmap, tgaFile.c_str(), 0);
return true;
}
@ { Angelius }
We haven't yet tried to reverse engineer this yet. My reverse engineering skills in this vein are rather rudimentary. I've poked around a bit, but am by no means knowledgeable enough to gleam anything useful. I have started to read up a bit and start teaching myself some of the aspects of it, but I think it'll be a minute before I am quite there. At most I found where the client identifies an RLE file by the ".RLE" extension, but after that it becomes to me a garbled mess! I was hoping to figure this out just by parsing the file data directly, but perhaps I need to bite the bullet in the long run...
I appreciate any bit of input and suggestions ya'll provide. I'm sure it'll crack eventually, despite those pessimistic moments of defeat.
Hey every, just wanted to give you a quick update...
I think I've nearly cracked it! Although my colors are off, the images are finally crisp and clean. The shadows are perfect, and the edges of the monsters are properly opaque. There are two issues to address. First, some of the images are flipped. There is probably a flag somewhere I am missing which dictates the direction of the bitmap. Second and most importantly, my colors are still off. This is likely due to my conversions of HSV to RGB. On that point, let me explain my findings...
1) Color map offset seems to be at about 22. I did a lot of brute force testing and tweaking here, and seem to be getting consistent results with that assumption.
2) The color map is stored in HSV not RGB! This really threw me for a loop, but when I noticed my R values seemed to drop around ranges of 32/96/128 for the turtles, which matches the expected values provided by the server hsv changes, it hit me that the RLE would be in HSV. This makes sense because, if you are going to modify the image anyways, why convert rgb>hsv>rgb and waste processing power, especially in 2001. Save the operations and directly store in HSV.
3) Since the pixel packet is two bytes, what does each byte mean? After playing around, I found one byte represented the alpha/opacity of the pixel. Making this assumption I found one of the bytes resulted in the perfect expected alpha. Everything not the image was clear, the edges of the monster and its shadow somewhat seethrough, and the inner pixels solid.
This is super exciting. Obviously the current colors you will see below are wrong. I am fairly certain the Hue is correct, so I will have to play with my Saturation and Value calculations some more and see what results I get.
MLE AND RLE is has been Detected 08/03/2009 - Grand Chase Philippines - 24 Replies detect na poh ung ating moonlight engine at radical
engine kailan poh mag
kakamerun ng bagong engine!!!??
请阅读 readme.txt 中的版权信息
See readme.txt for copyright information.
Description: RLE Sprite 头文件
Original Author: Cloud wind
Authors:
Create Time: 2001/6/1
Modify Time: 2001/11/12
.:*W*:._.:*I*:._.:*N*:._.:*D*:._.:*S*:._.:*O*:._.:*U*:._.:*L*:._.:*/
#ifndef _WINDSOUL_SPRITE_H
#define _WINDSOUL_SPRITE_H
#include "wsaddon.h"
#define _ERR_RLESPRITE 'r'
#define WS_RLESPRTIEFILEERR MAKE_ERR(_ERR_RLESPRITE,'FLE')
#define BLIT_ALTERNATE 0x1
#define BLIT_PALETTE 0x5
#define BLIT_EDGE 0x6
typedef WPixel* WPalette;
struct WSpriteStruct;
class WINDSOULADD_API WSprite : public WObject {
public:
WSprite() {}
WSprite(WPalette pal,void *data) { Create(pal,data); }
WSprite(const WSprite& sprite);
~WSprite();
void Create(const WPalette pal, void *data);
int GetW() const;
int GetH() const;
BLIT_FUNC DrawFunc(DWORD style) const; // 获得 Draw 函数指针
WObjStruct *Duplicate(WObjectHeap *heap) const;
WSpriteStruct *GetSpriteStruct() const { return reinterpret_cast<WSpriteStruct *>(objdata); } // 获取数据结构
void SetUserData(DWORD userdata) const; // 设置用户数据
bool IsCover(int x,int y) const; // 判断是否遮挡
bool IsCover(WPoint p) const { return IsCover(p.x,p.y); }
const WSprite& operator()(DWORD userdata) const { SetUserData(userdata); return *this; }
const WSprite& operator()(WPalette addpal) const { SetUserData((DWORD)addpal); return *this; }
WSERR Load(const char *filename,int *size=0); // 读文件
void *Zoom(int w,int h,int bSpecialPal) const;
};
class WINDSOULADD_API WSpritePak {
void *m_Data;
int m_Direction;
int m_Width;
int m_Height;
int m_Kx;
int m_Ky;
WPalette m_Pal;
mutable WPalette m_AddPal;
DWORD m_Size;
protected:
int m_Frame;
WSprite *m_Spt;
public:
enum { GreyPal=0, RedPal=1, GreenPal=2, BluePal=3, YellowPal=4, BlackPal=5 };
WSpritePak() : m_Spt(0),m_AddPal(0),m_Size(0) {}
virtual ~WSpritePak() { Destroy(); }
WSERR Destroy(); // 销毁
bool IsValid() const { return (m_Spt!=0); } // 判断是否有效
const WSprite* operator[](int dir) const { ASSERT(IsValid()); return m_Spt+dir*m_Frame; }
WSERR Load(const char *filename,int skip=0,int size=0); // 读文件
void InitAdditionalPal() const; // 初始化附加 Pal
int GetDirection() const { return m_Direction; }// 读方向数
int GetFrame() const { return m_Frame; } // 读帧数
int GetWidth() const { return m_Width; } // 读宽度
int GetHeight() const { return m_Height; } // 读高度
int GetX() const { return m_Kx; } // 读X
int GetY() const { return m_Ky; } // 读Y
DWORD GetSize() const { return m_Size; }
const WPalette LightPal(int level) const; // 返回亮度 Pal
const WPalette ColorPal(int color) const; // 返回彩色 Pal
private:
virtual void ReadAdditional(void *header) {} // 读附加信息
};
class WINDSOULADD_API WSpriteAni : public WSpritePak {
int *m_Delay;
mutable int m_CurrentFrame;
mutable int m_DelayFrame;
public:
struct Pair {
short current;
short delay;
};
WSpriteAni();
~WSpriteAni();
void Set(int frame=0);
void Set(Pair &pair,int frame=0) const;
int TotalFrame() const;
const WSprite& NextFrame(bool *change=0) const;
const WSprite& CurrentFrame() const {
return m_Spt[m_CurrentFrame];
}
const WSprite& NextFrame(Pair &pair,bool *change=0) const;
private:
void ReadAdditional(void *header);
};
#endif
