[Tool] edxNVMViewer Beta

[pushedx]

Here's a small update. I've been working through the BSR files a little because you have to use those files to get the BMS entries. I found some useful information that would easily allow for a rough dimension extraction from BSR files without having to go through the BMS files themselves. This dimension data is simply a AABB (axis aligned bounding box).

There are good things and bad things about this data. The bad things is that it's simply a maximal volume bounding data set, so objects will appear larger than their collision field might actually be. The good news is that it's ok since you couldn't feasibly generate a collision mesh efficient for player interactions unless you implemented your own model loader and had some simple physics for ray casting, so it all works out.

This means you can get the position of any BSR group (BSRs are like nodes of meshes) and get the dimensions that the total node takes up. You could then perform some secondary post processing on the data to change the size if it's a type of object you know will be too large, such as trees. Trees are huge and in game only their small trunks are collidable, so for any tree you can just use a fixed size that would best estimate the real size.

Here are some screenshots with the entities drawn to their AABB with trees set to a fixed size of 50 x h x 50. In addition, the entities are (should be) properly rotated to correspond to their in game rotations, which is very important!



Some of the buildings look wrong, so I have to double check some things. Originally, my coordinate system was flipped, so I have to go back and change how everything is rendered to get it to correspond correctly in game. For example, when you are in South Jangan looking out from town, the hill that is on your right in game was on the left in my original images. My layers are backwards as well.

Another issue is that since there are so many BSR files and it's relatively expensive finding entries based on their files names, what would be more efficient is simply going through all BSR files and saving their name and dimensions into a new data file rather than trying to read in real time from the PK2s. Trying to do it in real time adds like 30s - 60s for each region load and that's just horrible. Using new data file loaded into memory would be a lot faster.

Lastly, since the AABB data is just a bounding volume around a group of meshes, the information is better used to help avoid collisions or get "unstuck". Trying to use it all to create some automatic path finding system and dynamic collisions avoiding system in real time seems really impractical now. There is just too much data and you would need to make an offline program that would generate binary data for your program to use so it's more memory and processing efficient.

But even so, the rough data is "good enough" to draw on top of your 2D minimap and create tools that allow for path generation while being able to see obstacles to avoid! For now I need to redo the coordinate system used and figure out some minor inconveniences. I don't have plans on parsing the entire BSR files because you only need them if you wanted to properly render all the BMS files, and the BMS->OBJ tool has some bugs in there which would indicate a rewrite is needed to handle some extra cases. Something I'm not going to spend the time on now.

Right now, terrain height + nvm layers + entity positions + entity AABB is certainly "good enough" for making more advanced pathing and collision avoidance systems, so we can make the best of this stuff.

[xxbrentonxx]

Oh wow = 0. How do you propose Botsmall and SROKING team did their path finding systems?

[pushedx]

Originally Posted by xxbrentonxx

Oh wow = 0. How do you propose Botsmall and SROKING team did their path finding systems?

For Bot369's isrobot, I'd almost certainly say by hand. That's why isrobot follows the same path out of town to the training location. They do seem to have some simple collision avoidance, but it's flawed and that's reason why the bot wall humps and can get stuck easily.

You could create your own simple path finding system though by defining a large set of walkable points and then using a graph based path finding system like Dijkstra's algorithm. From there, you can easily go between any locations. I've made a simple system using this before and it's easy, but it just takes a bit of time to define all the walkable areas.

However, if you had a team of people working, it should be short work since the task can be split up. I prefer that approach myself because you can do some creative optimizations to speed it up and make it work really well. You would only have to spend maybe a couple of hours per world area (a lot more for caves/tombs though) and only have to do that work once. Trying to find an automatic way to do it only by processing files would be very tricky due to some things like bridges and ferries. You'd have to do those by hand.

I've not seen sroking in action to give a guess on how they did their system, but judging by how they have a lot of data files and text based files, I'm pretty sure they did it by hand too. A lot of bots just let you create your own walk scripts as well, which simplifies the task but creates more work for the end user.

Overall, I don't think there's anything really new in the NVMs that would just make anything possible that already is any more easier. I mean, considering how much data there is, you can program more advanced tools, but there is a lot of work that would still have to go into making your own path finding systems. This data certainly will help with the collisions though, and that is a big thing since you don't want your character getting stuck and unable to recover!

[JasonWalker]

OMG, you rock Drew!

[xxbrentonxx]

Originally Posted by pushedx

For Bot369's isrobot, I'd almost certainly say by hand. That's why isrobot follows the same path out of town to the training location. They do seem to have some simple collision avoidance, but it's flawed and that's reason why the bot wall humps and can get stuck easily.

You could create your own simple path finding system though by defining a large set of walkable points and then using a graph based path finding system like Dijkstra's algorithm. From there, you can easily go between any locations. I've made a simple system using this before and it's easy, but it just takes a bit of time to define all the walkable areas.

However, if you had a team of people working, it should be short work since the task can be split up. I prefer that approach myself because you can do some creative optimizations to speed it up and make it work really well. You would only have to spend maybe a couple of hours per world area (a lot more for caves/tombs though) and only have to do that work once. Trying to find an automatic way to do it only by processing files would be very tricky due to some things like bridges and ferries. You'd have to do those by hand.

I've not seen sroking in action to give a guess on how they did their system, but judging by how they have a lot of data files and text based files, I'm pretty sure they did it by hand too. A lot of bots just let you create your own walk scripts as well, which simplifies the task but creates more work for the end user.

Overall, I don't think there's anything really new in the NVMs that would just make anything possible that already is any more easier. I mean, considering how much data there is, you can program more advanced tools, but there is a lot of work that would still have to go into making your own path finding systems. This data certainly will help with the collisions though, and that is a big thing since you don't want your character getting stuck and unable to recover!

That would take ages to do it all by hand, anyway the reason it leaves town the same way could be because it has pre defined scripts to certain points.

[npcdoom]

Great project ^ ^

[lolrko]

Just 2 words to be said, Freaky Awesome .

[John Dread]

Originally Posted by xxbrentonxx

That would take ages to do it all by hand, anyway the reason it leaves town the same way could be because it has pre defined scripts to certain points.

Just wanted to contribute that they indeed do it by hand...got my sources :P

[maxbot]

Originally Posted by xxbrentonxx

That would take ages to do it all by hand, anyway the reason it leaves town the same way could be because it has pre defined scripts to certain points.

Actually, it wouldn't take ages; it's just a matter of patience. Creating paths all over the map with a proper in-house application is not a big deal, as long as you know what you are doing

[Ahmos2009]

Really nice I feel now I can make a BOT 4 psrvers but 1st I must go to learn vb & c++ I am not joking

U R The Man

[babalui]

I see "someone" is more than ready to get his own game up and running. Nice job Drew, as always

[pushedx]

Thanks

I'm hoping to get back to this project in the near future. I'll be cleaning up the code a little and adding some more docs so people can follow it easily. I no longer have a real use for it for the time being, so I'll try to get it out sooner than later.

[yagnesh_shenoy]

Awesome bro rofl i will show this to my friends and they will be surprised thxssssss

[pushedx]

This is a BETA release of my entire project. Please read everything first!

That means:

• There are still bugs
• The code is not optimized
• You still have to do a bit of work yourself to make use of this data properly

I no longer have any intentions of continuing immediate work on the NVM files because I am changing my Silkroad development focus now (going into design related tasks). I will eventually revisit this project, but it is no longer a project of any priority to me. That means this project will remain in a BETA state for a while. There are plenty of opportunities to make the code better or customize how you use it in your applications though.

Here's a brief overview of the project. You will need Visual Studio 2008 to be able to compile and run out of the box. All of the required 3rd party dependencies are included.

Blowfish.cpp/Blowfish.h - Blowfish library that I've been using in all my project for the PK2 api.

glm.cpp/glm.h - Not actually used now, but a good simple OBJ model loading library.

nvm.cpp/nvm.h - My new NVM library that will load NVM files from memory or from file. As you will see, there is not that much code to it and it's pretty simple. The tricky part was reversing it all though, so that's expected.

pk2reader.cpp/pk2reader.h - My new updated PK2 reader class that allows for reading PK2 files from "memory" rather than having to extract them to a file first. Very useful for this project.

pk2writer.cpp/pk2writer.h - My original PK2 writing class. I don't think I made any changes here.

SOIL.h - Simple OpenGL Image Library used for the DDS loading and rendering via OpenGL

Utility.cpp/Utility.h - Just some common functions I use.

edxNVMViewer.cpp - The actual program. It utilizes SFML for OpenGL setup and rendering. It's a bit of code, so it might take a while to follow through it.

Camera.cpp/Camera.h - Someone else's Camera library from a game programming site.

Now for some important notes:

1. The NVM code is almost 100% complete. There's still some fields I don't know what they are used for, but the bulk of the stuff is known.
2. The NVM rendering code is NOT complete. You have to do some data manipulation to render the terrain correctly as you will see in the code.
3. You need a BSR/BMS loader to get the entities rendered into the world. For now I have written a small function called GetAABBFromBSR which will return an AABB of the BSR being loaded.
4. The whole "logic" to loading the NVMs and the BSR data is really inefficient! This means it will probably run slow on some machines until it is optimized. The offending function is GetAABBFromBSR.
5. Having some game programming / OpenGL knowledge will help a lot with getting the NVM files rendered properly. My approach is "good enough" to show the main idea but it's not perfect.

There are two important paths to set for the project file that are set already, but in case you lose settings:


Finally, some last words about the project. First, make sure you read my 4 previous replies in this thread as they contain some good information.
, , ,

Now, with NVMs, you should be able to easily setup terrain height collisions on a server. You can use the rendering logic to make a mini client(less) as well since you can draw your character based on the current position it's at. You will need to understand the Silkroad coordinate system first.

The entities in the regions are a problem though. You know where everything is and have an AABB of the entity, which is nice additional information to have access to, but a BMS/BSR parser is really needed for a "complete" picture of the game. While most of the rotations seem to work right, some are wrong which means there are either flags that also affect how something is positioned or there is more data in the BSR that should be utilized.

Writing a NVM compiler should not be terribly hard if you base the code on the reader. I've not bothered with that aspect of the files yet, but one day I might if no one does it in the mean time. Right now simply understanding the files is what's really needed.

There's a lot of code and logic here, so expect to spend some time learning what is going on and studying the classes. Most people won't have a use for this stuff or data, but it's still interesting to know.

Here are the final screenshots:


Enjoy

[InvincibleNoOB]

Drew,Do you provide any license for your code? I'd like to use the nvm.cpp in a commercial project of mine.