PK2Export Beta Testing - Updated PK2 Extractor

Recently, I have been working on updating my old PK2 code. My new goals are to provide a new API that performs more efficiently than before, grants new functionality never before implemented, and all the while provides a simple and easy to use API for your own programs. This thread is the first beta test for my new API.

I have coded a simple C# application (.net 3.5) to use as a front end to my current API (PK2ReaderApi.dll). I have made the API a DLL for easy future updating when bugs are found. You will need to install the [Only registered and activated users can see links. Click Here To Register...] in order to run the program. Using the program is pretty simple:

1. Run "PK2Export.exe"
2. Use "File->Open" to select a PK2
3. Right click on an entry to extract it

More detailed information is available in the" _readme.txt" file. In this test, I'd simply like to get an overall response of how the application feels, does it seem to perform well, and of course, does it actually work for your extraction tasks. If you used any of my previous PK2 tools, you can compare them to those as well or any other existing PK2 tools out there.

I have included a brand new feature for testing, and that is PK2 defragmentation. Over time, PK2s that are patched a lot become fragmented with old file entries that are no longer used. This "wasted space" is never reclaimed due to how their PK2 code is setup. After so many patches, Joymax will simply redistribute new optimized PK2 files as part of their Legend installs (although that is due to how their DownloadServer is setup as well).

Using the "Defragment" option is really easy. Simple open a PK2 and select "Tools->Defragment". Hit Yes to start the process. Only media.pk2 should be defragmented on the official servers because the rest of the PK2 files are not patched often enough. In fact, you probably won't save any space for them and will just waste time. In the last screenshot, you can see how swsro would greatly benefit from this tool. I saved almost 300mb in unused entries from their files!

A new PK2 file is created in the program directory. To use it, simply rename it to the original PK2 name and place it in your Silkroad directory. Please backup your old PK2 files in case there are any bugs found! While I have done a bit of testing myself, more widespread usage is needed to determine if there are bugs or not. That is why I have new threads for testing programs and why a lot of my tools stay in the forever beta state. ;) I should note, I have tested using my defragmented PK2 with a version update on iSro and GFXManager.dll seemed to like it still, so I think it's going to be 'ok'.

Also included is a Blowfish key file system so you can easily use this extractor with other private server PK2 files. I have included the keys for mysro, swsro, and zszc in addition to regular Silkroad. Those are the only versions I could actually find up to date downloads for and test with. To switch the key, simply open "key.txt" and change the file name to the correct version you want. See my guide here for how to find the Blowfish key yourself: [Only registered and activated users can see links. Click Here To Register...]

That's about it for now, this program is a simple extractor to test the new API. Any bugs with the C# GUI aren't as problematic as problems in the DLL. It's important to take care of any possible bugs now before I move on to my PK2 writer API. Please test this as much as you can and post any results or comments. It would be greatly appreciated.

Screens:
[Only registered and activated users can see links. Click Here To Register...][Only registered and activated users can see links. Click Here To Register...][Only registered and activated users can see links. Click Here To Register...][Only registered and activated users can see links. Click Here To Register...]

Virus Scan: [Only registered and activated users can see links. Click Here To Register...]

Download: Attached (Binary only) Source project will be released after more testing + PK2WriterApi is done and tested. However, you can easily decompile the C# application to get the interface for PK2ReaderApi.dll and play with it yourself if you wanted to.

Thanks! :)

Edit: [Only registered and activated users can see links. Click Here To Register...]

#Approved

Looking forward to the source release. Your previous pk2 extractor performed twice faster than the first version. Hope this one handles the pk2 file faster.

I have fixed a few serious bugs that managed to slip through!

Quote:

0_2: Bug fix maintenance - Added an invalid iterator check that was missing - Fixed a few bugs where 0 byte entries might crash the API - Corrected an error where resources would be leaked on Close and a crash would result when trying to Defragment a newly opened PK2 - Corrected a C# GUI error where a dictionary was not being cleared on Close

Attached is the updated version. ([Only registered and activated users can see links. Click Here To Register...], [Only registered and activated users can see links. Click Here To Register...])

Has anyone noticed anything while using it?

Quote:

Originally Posted by pushedx I have fixed a few serious bugs that managed to slip through! Attached is the updated version. ([Only registered and activated users can see links. Click Here To Register...], [Only registered and activated users can see links. Click Here To Register...]) Has anyone noticed anything while using it?

Working perfectly for me

Thx Drew ;)

Did some more updating today!

Quote:

0_3 Update C# GUI - Removed hard file key system for user configurable key system. (could still use some work though) - Added configuration option for Defragment so a new key can be used. You can use this to change encryption from one key to another. - Added the current Key to the GUI - Added a menu item to open the output directory - Updated the C# gui logic to support non-encrypted PK2s (simply use no key) C++ API - Updated the PK2 reader api to support non-encrypted PK2s (useful for debugging, the client supports them as well) - Changed file open mode to allow read sharing. This was needed for my writer api and allowing Silkroad to use the pk2s while opened

Screens:
[Only registered and activated users can see links. Click Here To Register...] [Only registered and activated users can see links. Click Here To Register...] [Only registered and activated users can see links. Click Here To Register...]

From here, I am thinking about how to properly design my writer code to minimize risk of corruption while still utilizing existing code. I am looking into a way to build temporary update files then write all changes at the end. However, that complicates a few things I've already written so I'm going to spend some more time working out a better design that maintains performance without sacrificing safety.

Attached is the new version ([Only registered and activated users can see links. Click Here To Register...]).

the 0_3 gives me lots of critical errors

When open the editor:
[Only registered and activated users can see links. Click Here To Register...]

While loading a pk2:
[Only registered and activated users can see links. Click Here To Register...]

Not Work Win 7 :)

Thanks for the feedback guys. I noticed I had not set my project to target x86 only. When the x64 version tries to load the x86 DLL it will crash. I've fixed that first error shown in Kaperucito's screenshot. I forgot to test this on my x64 laptop so I missed that error.

As for the more crashes now on version 3, it seems to be happening on Vista/Win7 while not on WinXP. I've tracked down the error to the string wrapper functions I am using to pass C++ strings to C#. That method seems to break while not on x86 WinXP, so I've rewritten the code a bit. In addition, it seems the Visual Studio hosting process crashes the application in Debug mode, so I've removed that option as well. I'm not quite sure what's going on there, but I will look into it.

So, the problems right now aren't with the actual API backend but rather the interface DLL and the C# front end. I forgot about some of the issues that you have to account for when trying to pass pointers around and use the contents directly. The PK2Node pointers are fine because all access goes through the DLL for those.

I retested the new version on my x64 Vista laptop and x86 XP desktop and it seems stable now. One note to Vista/Win7 users is you need to run from a path that has write permissions so files can be saved to the current directory. I.e., if you aren't running as an Admin user and you try to work from a "protected" folder like Program Files, it might not be able to gain access to the folder.

Please try out this new version to see if your problems are fixed, thanks!

Quote:

Update 0_4b: C++ - Changed project settings to use Multibyte instead of Unicode - Added .def for API DLL for clean export names - Changed calling convention to stdcall instead of cdecl - Removed the system of returning strings from the DLL, strings are copied as they should into user buffers. - Removed extra debugging MessageBox on error C# - Changed target to x86 instead of any - Updated calling convetions to stdcall - Changed API functions to not return strings - Added a color code to the current PK2 Key to make it stand out. - Added a warning on PK2 open when no key was selected. - Added clarifications regarding the PK2 key on the defragment and change key dialog

Screenshots:
[Only registered and activated users can see links. Click Here To Register...] [Only registered and activated users can see links. Click Here To Register...] [Only registered and activated users can see links. Click Here To Register...]
[Only registered and activated users can see links. Click Here To Register...] [Only registered and activated users can see links. Click Here To Register...]

[Only registered and activated users can see links. Click Here To Register...]

Thanks for your attention. The other error got fixed, but now I'm getting this error each time I open any pk2:

[Only registered and activated users can see links. Click Here To Register...]

Quote:

Originally Posted by kaperucito Thanks for your attention. The other error got fixed, but now I'm getting this error each time I open any pk2:

You have to set the key first, so go to Tools->Change Key and select the key that best fits your PK2 set. You aren't the first person who has had this issue, so it's a usability error on my part.

I made the Change Key screen pop up the first time you run the program, but I guess I'll find a way to make it more "apparent" that there is no key currently selected.

However, sometimes users might not want to have a key select so they can work in unencrypted mode instead to be able open the output PK2 in a hex editor and they will be able to see the headers normally.

So that's why there is no key to start with unless you set it. Private server uses would set their key once and it'll default from there on and regular server users would set their key and it'd default from each startup. In addition, custom keys can be used of any length.

Edit: Updated to version 4b with these issues (hopefully) taken care of!

Wow thx nice, i have an suggestion for you.
Could you add the function "delete" to the extractor?
So we could clean up , mistakes which was wrong imported into the pk2 :)

Quote:

Originally Posted by _Jefrey_ Wow thx nice, i have an suggestion for you. Could you add the function "delete" to the extractor? So we could clean up , mistakes which was wrong imported into the pk2 :)

awesome idea!