How to manage Nostale map

05/17/2022 17:52 Apourtartt#1
Hello

One thing I was doing in early 2021 was a Map Editor.
I never finished it for two reasons. One being described later.

First, here is the link to the git repo: [Only registered and activated users can see links. Click Here To Register...]
and here is what it currently look like: [Only registered and activated users can see links. Click Here To Register...]

The explaination of what it is is in the README. This topic will be useful for those who are really motivated to create a map viewer/creator - copy pasting won't be enough : there are still a lot to do.

After reading the README, you know the limitations of the blender script. So here we go :

First, how NStuData.NOS works :
It is, as a lot of other .NOS files, just an archive, you can read it "humanly" once you know how it is written, let's take a look.

[Only registered and activated users can see links. Click Here To Register...]
I am using HxD in order to have the hexadecimal (left part) representation.
Red part: file header (it will always be the same for NStpuData, it will be another for NStgData, etc)
Orange part: no clue, sorry
Green part: Number of files in this archive - note that it is in little endian, it means you have to read it from right to left, instead of reading 0xBA020000 (3120693248 files), you read it 0x000002BA (698 files)
Blue part: no clue, sorry

[Only registered and activated users can see links. Click Here To Register...]
Red part: ID - starts from 0 to... number of files - 1 (0 to 697 in my case) - for this file, it is also the map ID
Green part: Offset. If you don't know what an offset is, it is just a position relative to something.
(Note, there are number of files * 4 * 2 pairs, if you have any programming knowledges, just think of a loop)

Example :
My offset is 0x000015E5 (remember, little endian) - it means that if I go to 0x15E5 (we can ommit the left 0), we will have our data :
[Only registered and activated users can see links. Click Here To Register...]

Obviously, we are not expecting to have only 0x10 as data.
This is what we are left with:
[Only registered and activated users can see links. Click Here To Register...]
Red part: I don't know
Green part: data size (how many bytes the map data is coded on)
Blue part: packed data size (you can think of a .rar size (blue part) vs the un-archived size (green part)
Purple part: Wether data is packed or not.
The rest : data.

If purple part is 0, it means it is not packed (so green part = blue part, by definition), if purple part is 1, it means it is packed.

If you know some archiving algorithm or whatever it's called, you can maybe recogniez what is used by looking at the following bytes (0x78) - it is Zlib.
So you can just use Zlib algorithm in order to unpack the map, from 0x78 to where 0x78 is + blue part
Example : 0x78 is at 0x15F2 and blue part is 0x4F4A if we sum them up, the result is 0x653C, so we should un-zlib from 0x15F2 (included) to 0x653C (excluded).

Then, just repeat it.



In Golang, you can parse it that way:
Code:
func Unzlib(data []byte) ([]byte, error) {
	var out bytes.Buffer
	bb := bytes.NewBuffer(data)
	r, err := zlib.NewReader(bb)
	if err != nil {
		return []byte{}, err
	}
	defer r.Close()
	_, err = io.Copy(&out, r)
	return out.Bytes(), err
}

func unmarshal(data []byte) (FileNtData, error) {
	var fnt FileNtData

	if len(data) < 0x15 { // Be sure the file is larger than header
		return fnt, errors.New("file is too short to be NT Data")
	}

	fnt.header.header = data[0:0x0C]
	fnt.header.unknown1 = data[0x0C:0x10]
	fnt.header.nbFiles = binary.LittleEndian.Uint32(data[0x10:0x14])
	fnt.header.unknown2 = data[0x14]

	if uint32(len(data)) < 0x15+0x8*fnt.header.nbFiles { // Be sure the file is larger than header + entities info
		return fnt, errors.New("file is too short to be NT Data")
	}

	for i := 0; i < int(fnt.header.nbFiles); i++ {
		var d ntData

		d.id = binary.LittleEndian.Uint32(data[0x15+(i*8) : 0x19+(i*8)])
		d.offset = binary.LittleEndian.Uint32(data[0x19+(i*8) : 0x1D+(i*8)])

		fmt.Println(d.id, d.offset)

		if len(data) <= int(d.offset)+0x0D { // Be sure the file is larger than where data is + data info
			return fnt, errors.New("file is too short to be NT Data")
		}

		d.unknown = data[d.offset : d.offset+0x04]
		d.dataSize = binary.LittleEndian.Uint32(data[d.offset+0x04 : d.offset+0x08])
		d.packedDataSize = binary.LittleEndian.Uint32(data[d.offset+0x08 : d.offset+0x0C])
		d.isPacked = data[d.offset+0x0C] != 0

		if len(data) < int(d.offset)+0x0D+int(d.packedDataSize) { // Be sure the file is larger than where data is + data info + data
			return fnt, errors.New("file is too short to be NT Data")
		}

		d.data = data[d.offset+0x0D : d.offset+0x0D+d.packedDataSize]

		if d.isPacked {
			var err error
			if d.data, err = cryptoutils.Unzlib(d.data); err != nil {
				return fnt, err
			}
		}

		fnt.data = append(fnt.data, d)
	}
	fnt.sort()

	return fnt, nil
}
By repeating those steps (or by using this code), we are left with the map structure files.
Here is what it looks like :
[Only registered and activated users can see links. Click Here To Register...]
It is the same principle: there are some bytes and we need their meaning - there are at least two ways of doing it, I will explain both later, let's first about known values.

[Only registered and activated users can see links. Click Here To Register...]
Any part that is before the square is not important (it is metadata for the map, that you can see there: [Only registered and activated users can see links. Click Here To Register...])
The black part is the number of UNIQUE models which are imported.

Then comes a list of ids (4 bytes each)
[Only registered and activated users can see links. Click Here To Register...]
Red part: model id - it is basically the name of the .obj. For example, here it is 0xA22 (2594), it means:
[Only registered and activated users can see links. Click Here To Register...]
(Note that I am working on the map id 63)
Blue part: Another model id
Red : same, etc

Then we go to 0x89 + nb of models * 4, in my case, 0xC5 :
[Only registered and activated users can see links. Click Here To Register...]
(one at the right, in reality - it starts at the 0x01)
This is the start of an object. In this block, everything related to it is described: its color, its position, its rotation axis, its size, etc.
[Only registered and activated users can see links. Click Here To Register...]
The first selected byte (0x01) is what I called "interaction type". It defines two things :
1) the space the object takes in memory, 0x01 means it will take 0x45 bytes - see [Only registered and activated users can see links. Click Here To Register...] for more informations
2) Not 100% sure - it also defines what kind of object it is.
0x00 seems to be useless
0x01 means it is "ground"
0x02 means it is an object, like a rock, etc
0x03 means a light or something like that (or I confused with 0x00)

[Only registered and activated users can see links. Click Here To Register...]
Red part: model index - in this case it is 0x0000, meaning the first one, so we go at 0x87 + this value. 0x87+0x0000 = 0x87, so we take a look at the value, it is 0x00000A22 (2594) it means it is the object I showed you earlier.
Orange part: X position
Green part: Z position
Black part: Y position

I lost some data, so you will have to continue by yourself, I will explain two ways of doing it.

First: bad but easy method (I recommand starting by this one)
- Open the map file with an hex editor (HxD for example)
- Go to a data block that codes for an object (the blue selection on the screen above for example)
- Change some data (I recommand 1 by 1)
- Save, import it on OnexExplorer, save the NStuData file, place it in Nostale/NostaleData directory
- Connect to Nostale, go on the map and see the changes (and note it, obviously)
- Revert your changes and start again with another offset

Second:
- Connect to Nostale, go on the map
- Start Cheat Engine as admin, attach it to Nostale
- Open the map file with HxD
- Pick up an object you would like to observ
- Copy some data on it (ex :
[Only registered and activated users can see links. Click Here To Register...]
)
- Save the address value (0xEB7EFDC in my case)
- Open Reclass as admin: [Only registered and activated users can see links. Click Here To Register...]
- Attach Reclass to NosTale
- Enter the address in the """textbox"""
[Only registered and activated users can see links. Click Here To Register...]
- Change the address that way : remove the last digit/letter and change it by 0 : 0xEB7EFDC -> 0xEB7EFD0
- Change the address that way : remove the before last digit/letter by 1 : 0xEB7EFD0 -> 0xEB7EFC0
- Repeat until you see something like :
[Only registered and activated users can see links. Click Here To Register...]
You MUST see a class name when hoverring the address "NostaleClientX.exe+something"
In my case, I had to reduce from 0xEB7EFD0 to 0xEB7EFA0 in order to see this. It means every values below this line is a property of our variables (well, not if you go too far)
- Now, go back on HxD and on Reclass and compare the values:
[Only registered and activated users can see links. Click Here To Register...]
(I recommand taking a screenshot of Reclass before any modifications in order to have a backup)
- Then go back on Reclass, and change some values - for some values you might change map or move camera in order to notice the difference, for some you won't need.
Here is an example:
[Only registered and activated users can see links. Click Here To Register...]

Also, I created long time ago that can help :
- When you try to add an object, it won't work, you have to replace one, so there is something that tells the number of objects on the map or something like that
[Only registered and activated users can see links. Click Here To Register...]


I hope this was useful, I think every important details have been discussed and you should be able to link data to what they do and then be able to continue this work!
05/17/2022 18:42 Bejine#2
Quote:
Originally Posted by Apourtartt View Post
Red part: ID - starts from 0 to... number of files - 1 (0 to 697 in my case)
that's not true, ID can be anything (as long as its unique, and i think it also needs to be sorted), it doesn't have to be continuous from 0.
05/17/2022 19:57 Apourtartt#3
Quote:
Originally Posted by Bejine View Post
that's not true, ID can be anything (as long as its unique, and i think it also needs to be sorted), it doesn't have to be continuous from 0.
You are right, but if I go into details of everything I would need more than 2 hours to write it :(