Mabinogi File Formats Documentation

[adam_j]

I wasn't sure whether I should bother to post this here, because well, nothing of value ever really takes place here.

But what the hell, maybe you guys will surprise me.




So, I've been staring at Mabi's files in hex for a week now, and examining source/output of various converters, and this is what I have so far, on PMG files: (There's more, but I've literally only just finished typing the PMG section so far, which is by -FAR- the section I know most about)

I'll start with some basic grouping conventions that are applied in many of the formats, before going into specific detail on any of the formats:

All files formats start with some sort of definition in the first 3 bytes, take a look, you'll see what I mean.

Names take up at least 32 bytes usually. Some can take upto 128 bytes, and the client will probably expect it to be that way, but you can usually see quite easily what sizing format it's using (The relevant value is usually followed by a sea of zero bytes). This seems to be part of Devcat's cunning plan to be as inefficient as possible.

If you see 00 00 80 3F, you're in the Matrix. Well, A matrix. I fully recommend that you stop consuming whatever pills you're taking immediately.
Seriously though, these are usually Frame Transform Matrixes, you'll often find three 00 00 80 3F values in a matrix, punctuated by a number of 00 00 00 00 groups. These come in several flavours, 4 bytes x 16 groups being the most common, and most often a Frame Transform Matrix.
Basically, these tell the renderer how to size the object. The 80 3F is a floating point value of 1.0, so it means that it's treated as a 1:1 ratio.
Try altering these values for amusing results.

That's really all the non-specific things I can think of, and as I'll explain the files in detail (to the extent that I CAN), it's simply there for reference.
Don't fret if you just read that, and you're looking at your screen like "......O__o..?", It'll make sense in a bit.

So, to begin with, the file type I know most about:
PMG:

Before you start, download this:

 Spoiler

This is an example using the ninja suit.pmg in the archive. If you examine it in a hex editor as you read along, it will all make sense.

Starting at the first byte, the first 10 bytes are important to defining the file, the first 6 telling you what kind of file it is, and what version.
For instance the included PMG has the following bytes "70 6D 67 00 02 01". This tells you that it's "pmg-2.1".
The immediately following two bytes are usually relevant file information, for instance, these two are "D2 11", which is Little Endian for 11D2. This number, after the uniform 2 zero bytes, will take you directly to the end of the data for that segment.


I'm not entirely sure what most of the data is for in this segment, but I'll note down what I DO know.
After the initial D2 11 and following two zero bytes, you can see the name for the PMG, which takes up 128 bytes. In some PMGs, it seems to contain floating point values, but they don't seem important.
The 05 you see here is for SOMETHING, and it's relevant, (any higher and it crashes the PMG viewer, any lower and it doesn't display fingers/hands or at 0, anything). If anyone works out exactly WHAT it's for, please let me know. I'm assuming it's a four byte value, as the viewer crashes with data in the following three bytes.
The next 64 bytes have no value in any PMG I've examined so far, so I guess we can assume for now that this is filler.
The next byte seems to be related to the 05 from earlier, in some PMGs, like for pets, it's equal to the number of every part in the PMG (and the earlier value is 01, probably in regards to what I assume is the PMG equivalent to COM), and if the value of both is greater than the number of parts + 1, it crashes the viewer.
I've tried editing my ninja suit to be 01 and 14, but the only values it accepts are 01 and 13, which don't show all the body parts. Again, if anyone works out what the deal is with these, please, let me know, as I need this information to finish my 3DS Max plugin.
In the provided PMG, if you scroll down to 0x176 (the second "body") you'll see that there's 32 bytes from "body" to the next value, 01, which is followed by a seperating double zero byte from six bytes.
The first four I'm not certain of the function of (I've not gone and gotten myself a Ninja suit, or modified a PMG of something I actually have, so I'm not entirely sure if the client CARES about the values in these bytes), but the following two are a direct reference to the segment of data for that portion of the file
(search EA 05, the only two occurances you'll find, are the one you started at, and at the start of the actual information for that part), and as such, are required even by the PMG viewer.
Some data in files is completely ignored by the client, these values may be treated the same way (There's apparently Korean in the files, but I can't say whether it makes sense or not).
As you can see, some of the data here is obvious, like "male_cloth01_bss_body__" (Defining the texture for that part).
None of this data is particularly relevant for someone making a custom PMG for the game, as the actual visual data starts from 11D2 (Observant people will notice something here), from the point "pm!-1.7". As you might have already guessed, this version information is going to be a recurring theme.
To my knowledge, PMG files contain either 2.0 or 1.7 pm! data. I'm only explaining 1.7 here, as I've not really looked looked at any 2.0 data, but it should follow the same conventions as the 1.7 data, with minor changes. (If that's not the case, feel free to post with your findings.)

So, you've finally made your way to the part of the file containing the actual visual information. As you can see, it's broken up into 20 parts, each for a different body part.
There's always 10 bytes from the 70 to the second 0 byte, which is followed by the name of the part you're going to attach this segment of the PMG to, in this case, "body".


From the initial byte containing 62 (or b), there's 32 bytes that are reserved for the name of the body part.


The next 128 bytes contain the name of the PMG part (Open the PMG in the PMG viewer, click one of the parts, and you'll see what I mean).


Again, another 32 bytes for the part containing "body", ( ) and then followed by 32 bytes of zero value ( ), which is followed by ANOTHER 32 bytes of zero data ( Not sure what caused that white patch).

The converter source I've checked suggests there might be data at these points, but I have yet to see any, personally, but for the sake of completeness, there you go.
Here you see the value "c2", which is the colour channel for this segment. This takes up 32 bytes (Seriously..?) so, just skip those empty bytes again.

And now, it's time for the good crap.
Frame Matrixes.
These are all floating point values, this particular one starts with four 0 bytes, and contains 16 x 4 byte groups. So, the next 64 bytes are all related to the current frame matrix, which is the Minor Matrix.
The minor matrix isn't really relevant for our exported file, at least, it's commented out in all the .x files (I know.) I've seen so far, simply being included for the sake of completeness it seems.
I'm still decyphering parts of the Java PMG editor at present, so I may find out something interesting about it at a later stage. (It's probable that all commented out frames/mesh data are related to joint data, as the MQO converter skips those, and has broken joints.)

Another frame transform matrix, this time, the Major Matrix.
16 x 4 again, this one isn't commented out, and is pretty hilarious to mess with (try changing an 00 00 80 3F to 00 00 00 40 and loading the PMG in a viewer xD)

The following 12 bytes define the number of parts, in this case, 1.

The next 32 bytes contain the texture of the PMG, "male_cloth01_c", and a crapload of zero bytes. (no idea what's going on with the white patches..)

The next 116 bytes are simply a repeat of the previous information ( ), which is followed 8 bytes later by a matrix the converter lovingly refers to as "What Matrix", implying that the creator didn't know what it's for. This particular matrix is 15 groups of 4 bytes (According to the converter source anyway, I suspect it includes the 40 00 00 00, due to the fact that every other matrix is 64 bytes), and not necessary to export the PMG visual data, for editing. It's possibly joint data, however.


Following on from this, the next 192 bytes are the Mesh Normals for the frame, but for some reason, the other 134 bytes are commented out in the .X file.
Even the PMG/MQO converter has bytes that the other converter comments out.

From 1A56, we have several bits of data, combined.
The first three 4 byte values are Mesh Vertices, the next 3 are the Mesh Normals, followed by Infinity in float format. The next two 4 byte floats are Mesh texture co-ordinates.
And repeat, 28 times.

The last 192 bytes, [s]I have literally no clue about, other than they're likely floating point values (00 00 80 3f = 1)[/s] are possibly Face Size data, from reviewing the converter output.

And after that, you can see that another part of the PMG file.

Congratulations on managing to read that wall of text!

The simpler version:

 Spoiler

This is an example using the ninja suit.pmg in the archive. If you examine it in a hex editor as you read along, it will all make sense.

File start:
First 6 bytes: File type and version number
Following 2 bytes tell you where the visual data starts.
2 zero bytes, then 128 bytes for the PMG name.
The following byte is important in regards to displaying the model, but I'm not entirely sure what it's supposed to be for, however, I'm certain that it's a 4 byte value (Having messed with this section.)
The next 64 bytes are blank, and the following 4 bytes are, like the previous 4 I mentioned, important to displaying the model, but I'm not sure how/why yet.
The next 2 (possibly 4) bytes are a reference to the visual data, which I'll come back to.
32 bytes for the name of the body part, 128 bytes for the mesh name, and another 32 bytes for the body part, again.
I think this is where the next mesh part is defined, so I'll just outline two last things:
I believe the 01 here is a reference to the number of parts to the mesh, and the following 3 bytes are empty.
The next 4 bytes I have no idea about, but the 2 following it are essential to displaying the mesh correctly.

Now for the visual data:
If you scroll down to 11D2, you'll see "pm!" followed by 01 07.
This is the version data for this part of the mesh. To my knowledge, there are only 1.7 or 2.0 meshes, and all PMG files seem to be 2.1.
Immediately following that is 8408, which you'll likely recall from the first section of the file. The parts all have these numbers in the headers, and they're the number of bytes to the end of that segment of mesh data, not including the 2 zero bytes.
So, 32 bytes for the body part name, 128 bytes for the mesh name, and another 32 bytes for the body part name.
There's then 2 x 32 bytes of empty space, which the converter source I've read (included in the attached archive for anyone who is interested) seems to imply that there might be data in these segments in other files. I've not seen it myself.
32 bytes for the colour channel.
Here we have a frame transform matrix, 4 bytes x 16 values. I'm not sure what it's for, but the converter source calls it the "Minor Matrix".
Another matrix, 4 x 16, this one is dubbed the "Major Matrix", and is relevant enough to not be commented out in the .X file output.
The next 12 bytes are the number of parts.
32 bytes containing the texture of the mesh.
The following 120 bytes contain information, the number of parts in the mesh, the number of face vertexes, face count, "xxxxVertex", "xxxxCount", mesh count, and skin count.
If you run a file through the converter in the "PMG Exporter-etc" folder, all this information will be there, commented out.
There's another matrix here, but the converter calls it the "WhatMatrix", and says that it only has 15 values. I personally suspect that the 40 00 00 00 is actually a part of this matrix. Not sure what it's for.
The next 326 bytes are the face information, 134 bytes of which get commented out by the afforementioned converter.
From 1A56, we have several bits of data, combined.
The first three 4 byte values are Mesh Vertices, the next 3 are the Mesh Normals, followed by Infinity in float format (This is probably the mesh vertex colours information). The next two 4 byte floats are Mesh texture co-ordinates.
And repeat, 28 times.

The last 192 bytes, I have literally no clue about, other than they're likely floating point values.
That's pretty much everything.

PMG related files:



The point of this, is quite simple: Custom content.

Several private servers are in development, and with documented file formats, new monsters, pets, clothes, even new races, with their own frames, and working joints, could easily be created.

Also, feel free to PM me if you want to get involved in the project, or just post any contributions you might have.

[adam_j]

Since I was updating a thread elsewhere, I may as well add this here too:


Frm files:



The first 3 bytes tell you that the file is a Pleione framework, and that apparently is exciting, due to the the exclamation mark xD;

Bytes 5 and 6 tell you that the frame is version 1.0, and byte 7 tells you that there are 26 bones in this frame.


The blueish/purple line outlines the location data for the "com" bone (this bone is the root, basically), with all values being float points. I'm not really sure what the values do, as of yet, as I have yet to experiment with them all. This section is 192 bytes, so, possibly 3 transform matrixes?

The green line outlines the bone's name, which is 32 bytes in size.

The red line shows the bone's assigned number, and what it's connected to. Com is bone 00, and is connected to FF. The following 2 bytes are simply empty.

The light blue line outlines 32 bytes of data that the client doesn't care about. At all. I deleted all 26 sections, the client doesn't appear to even notice.

That's the format for the bone information, which is simply repeated 25 more times, with different information in the sections ^^


Now, the part I don't understand about this file is the remaining 50%:


Red outline: number of bytes in this section. (9348, not including itself. Always tells you the number of bytes.)

Green outline: Number of bones in the frame (Always.)

Light blue: 2x the number of bones in the frame (Always.)

Orange: 18? 4608? A floating point? I have no idea (Not always 18. Also not sure if the yellow section following is a part of it.)

Purple: Again, not sure what this is, or what it does, or even if the yellow outlined parts that follow are a part of it. (I assume that they are, though.)

Pink: The number of visible equipment slots. A character has 9 visible pieces of equipment, altering that to 08 made all equipment in my hands invisible (Which was pretty amusing.)

Grey: The client just doesn't seem to care about these bytes. I filled them in with random junk data, client didn't even notice. The other values are essential though.

Everything else: Data involving the display of the body parts. I've tried filling bytes in with zeros, it takes a LOT of zeros to make a difference, and as such, I've not worked out any sectioning to this data.

Perhaps someone here would be able to shed some light on what this data could mean?
Also, do the pictures help, or not?

[Somebody Fake]

Stop copy pasting the thread from MM. Why not just link to it?

[adam_j]

Originally Posted by Somebody Fake

Stop copy pasting the thread from MM. Why not just link to it?

Because not everyone HAS an MM account, and I'm not updating it for either site specifically anyway.

But you are indeed correct, posting it on either site serves zero development purpose, I'm simply doing so out of interest.

[harrybong1]

Pictures help, i like the multi colored lines. What are you hoping to achieve with this adam?

[adam_j]

[quote=adam_j;16755047
The point of this, is quite simple: Custom content.

Several private servers are in development, and with documented file formats, new monsters, pets, clothes, even new races, with their own frames, and working joints, could easily be created.
[/QUOTE]

Originally Posted by harrybong1

Pictures help, i like the multi colored lines. What are you hoping to achieve with this adam?

The coloured lines were totally not just an excuse to play around in paint, instead of look at hex.

<.<

>.>