[WIP] Silkroad's GFXFileManager API

[PortalDark]

Originally Posted by Nezekan

Everything is possible, but I wonder why you would want to do that. The files need to be loaded in as quickly as possible, and compression algorithms are certainly not built for speed, the client isn't that big anyway, the only reason you might want it to be smaller is for distribution but you'd not need this project for that

lets say for smaller distribution
yes, you got a point there: speed issues

[kevin_owner]

Originally Posted by PortalDark

lets say for smaller distribution
yes, you got a point there: speed issues

if you want to make the size of the client smaller you could remove all the unused stuff in the client. There is actually A LOT of data which can be removed without consequences.

Someone told me that about 30% of the .nvm files are old or probably never even used. These navmesh files doesn't have a related minimap image and are completely empty if you spawn in those ingame. But still they are 100 mb in size of not more.

And then there are all these ingame 3d objects who knows how many aren't even used.

Also for distribution it doesn't matter if you compress the pk2 files or not. If you pack to a .rar archive it'll be arround 1.5 gb. So if you compress the pk2 files you won't gain much once you create the final archive for distribution.

Owh and if you want to make the .pk2 files smaller once installed you could use the defrag tool in drew's 5 in 1 pk tools bundle

[PortalDark]

Originally Posted by kevin_owner

if you want to make the size of the client smaller you could remove all the unused stuff in the client. There is actually A LOT of data which can be removed without consequences.

Someone told me that about 30% of the .nvm files are old or probably never even used. These navmesh files doesn't have a related minimap image and are completely empty if you spawn in those ingame. But still they are 100 mb in size of not more.

And then there are all these ingame 3d objects who knows how many aren't even used.

Also for distribution it doesn't matter if you compress the pk2 files or not. If you pack to a .rar archive it'll be arround 1.5 gb. So if you compress the pk2 files you won't gain much once you create the final archive for distribution.

Owh and if you want to make the .pk2 files smaller once installed you could use the defrag tool in drew's 5 in 1 pk tools bundle

kgb packer can compress 1gb into 10-15mb of data(i tested with other MMO client, it takes around of 10h to compress it but to unpack, its pretty fast)
nut thats not the point
to clean the client of unused stuff, you need patience, time and lots of reading hours

[pushedx]

Sorry about the delay in my replying in this thread. I was going to make another post soon with something different for this forum, as I did post something on RZ already, but I have been really busy lately.

Originally Posted by kevin_owner

About the new formats instead of using the .pk2 files. Can't you just call the GFXFileManager's method's to get the files you want? and if they decide to change the file format you just copy the new GFXFilemanager?

If you implement a new format and don't account for that, then yes, someone could just use the new DLL's API to easily obtain the files.

Making a new file format to try and protect files to prevent people from getting access to them is really complex, as you need to change GFXFileManager.dll, sro_client.exe, Silkroad.exe, as well as implement another custom security DLL to enforce the changes.

There's a lot of possibilities though, but it'd be another tl;dr post to talk about that in depth.

Originally Posted by PortalDark

pushedx
with this kind of API, could it be used to compress even more the client?(lets say using a compressing method like 7z)

Yes, you can implement LZMA compression for the data files, but you have to take into account the extra processing overhead required at runtime. There are trade-offs for everything, but it is certainly doable. As part of another post I'll do, I'll cover this a little more. For as many files as there are on Silkroad, zlib would probably be better as a persistent file format if you intended on leaving the files compressed due to a lot less processing overhead.

Originally Posted by PortalDark

kgb packer can compress 1gb into 10-15mb of data(i tested with other MMO client, it takes around of 10h to compress it but to unpack, its pretty fast)
nut thats not the point
to clean the client of unused stuff, you need patience, time and lots of reading hours

On my RZ thread, which I can't link here, jMerlin and myself talked about a cool idea using HTTP processing. The idea would be that you distribute a client with the minimal amount of files, then allow the client to simply download the files it needs as they are loaded. What would result would be a minimal distributed client, strongly compressed files that are sent over HTTP (which allows for cloud hosting / static content caching), and the ability for people to try out new servers without having to download all the extra useless stuff that isn't even used. There would be certain technical issues to overcome, but it is something that would be worth looking into.

[giniyat202]

nice keep it up

PS: i dont suggest putting every thing in only one cpp file.. it makes everything hard to understand :S

[Tyră]

I wish I'm sure that's failing, but nice: D

[Yo123]

were you able to fix the white textures yet? i remember sro client having extra references for textures.. was doing the same some time ago for satisfaction during the time it lasted.

[pushedx]

Originally Posted by Yo123

were you able to fix the white textures yet? i remember sro client having extra references for textures.. was doing the same some time ago for satisfaction during the time it lasted.

Yes! I haven't updated this thread yet or had the time to implement some of the other stuff I wanted to like a compression examples, but basically it's all done. I have another thread posted on a different forum where I thought I'd get more testers so if you search for "GFXFileManager-0.0.1", you'll find the thread.

The problem was with file path characters. I had to cleanup some of the client requested paths to be compatible with the way I was accessing the files. In addition, I fixed a few other minor bugs in the code, but so far from the testing done, it seems pretty solid.

There are certain performance issues with files that are unpacked like this, depending on your computer specs, but it's not a huge deal for development.

I'm still contemplating if it's "worth it" though. I've been following the current state of sro pservers, and I'm not really inclined to spend a ton of time on this stuff really. Most of the creative ideas I have for Silkroad now rely on web based interactions, which seems pointless at the moment with all of the web attacks going on. Maybe when things settle down some, I'll come up with some more PoC projects.