Register for your free account! | Forgot your password?

Go Back   elitepvpers > MMORPGs > Rappelz > Rappelz Private Server
You last visited: Today at 16:06

  • Please register to post and access all features, it's quick, easy and FREE!

 

[RELEASE] DataCore v3

Reply
 
Old   #1
 
elite*gold: 0
Join Date: Jul 2015
Posts: 206
Received Thanks: 223
[RELEASE] DataCore v3

Current Version

v3.0.0.4


Introduction

Over the past few years many developers have released tools/source aimed at client editing, be that client dumping, rdb editing, client patching and more. During all this time however, none have ever developed a project-centric approach that would easily allow any individual to incorporate such functionality into whatever project they desire. Be that a launcher, a dumper, a patcher etc.. Well today I'd like to introduce you to a project I've been working on behind the scenes for some time...DATACORE v3


Features
  • Overridable Encoding (with default)
  • Create data.000 from dump path (expects extension structure e.g. dump/tga dump/jpg etc..)
  • Load data.000
  • Save loaded/created (caller provided) data.000
  • Get entry/entries from data.000 based on name, dataId or extension
  • Delete entries from data.000 by dataId
  • Export file/files from data.000
  • Update file/files in data.000 (with new versions of locally stored files)
  • Import file/files into data.000
  • Build data file/files (data.xxx)
  • Get dataId of file by fileName (hashed or unhashed)
  • Get if fileName is hashed
  • Event driven and in-depth progress reporting
  • In-House error handling with Event driven reporting
  • Progress reporting via events


Usage

DataCore is intended to be embedded into any .NET 4.5.1+ project. See below example:

Code:
Example:

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using DataCore;
using DataCore.Structures;

public class Example
{
	Core core = new Core();
	List<IndexEntry> index = new List<IndexEntry>();
	
	public Example()
	{
		core.WarningOccured += (o,x) => { MessageBox.Show(x.Warning, "Warning Occured", MessageBoxButtons.OK, MessageBoxIcons.Warning); };
		core.ErrorOcurred += (o,x) => { MessageBox.Show(x.Error, "Error Occured", MessageBoxButtons.OK, MessageBoxIcon.Error); };
	
		// The following assumes your program gui has two progress bars one for Total events and one for Current events
		core.TotalProgressDetermined += (o,x) => { TotalProgress.Maximum = x.Maximum; };
		core.TotalProgressChanged += (o,x) => 
		{
			TotalProgress.Value = x.Value;
			//TotalStatusLabel.Text = x.Status;
		};
		core.TotalProgressReset += (o,x) => 
		{
			TotalProgress.Value = 0;
			TotalProgress.Maximum = 100;
			//TotalStatusLabel.Text = string.Empty;
		};
		
		core.CurrentProgressDetermined += (o,x) => { CurrentProgress.Maximum = x.Maximum; };
		core.CurrentProgressChanged += (o,x) => 
		{
			CurrentProgress.Value = x.Value;
			//CurrentStatusLabel.Text = x.Status;
		};
		core.CurrentProgressReset += (o,x) => 
		{
			CurrentProgress.Value = 0;
			CurrentProgress.Maximum = 100;
			//CurrentStatusLabel.Text = string.Empty;
		};
	}
	
	public async void LoadIndex(string indexPath)
	{
		if (File.Exists(indexPath)
		{
			await Task.Run(() => { core.Load(indexPath); });
		}
		else { /*File doesn't exist!*/ }
	}
	
	public void FindEntryByName(string fileName)
	{
		IndexEntry entry = core.GetEntry(ref index, fileName);
		
		if (entry != null) { long fileOffset = entry.Offset; }
	}
	
	public void FindEntriesByDataId(int dataId)
	{
		List<IndexEntry> entries = core.GetEntriesByDataId(ref index, dataId);
		
		if (entries.Count > 0) 
		{
			foreach (IndexEntry entry in entries)
			{
				/* do work */
			}
		}
	}
	
	public async void ExportAll(string clientPath, string buildPath)
	{
		await Task.Run(() => { core.ExportFileEntries(ref index, @"C:\Rappelz\", @"C:\Dump\", 0); });
	}
}

Installation

Note: DataCore is written in c# for .NET 4.5.1+

In your .NET solution (.SLN), navigate to your project on the solution tree explorer. Right click 'References' and click 'Add Reference', now click 'Browse' and navigate to where you have placed the DataCore.dll (doesn't matter where it is, when you build the project it will normally be placed beside your .exe) Now simply add the assembly reference on any .cs you wish to use DataCore's functionality with: using DataCore;

Note: The DataCore.dll is heavily summarized (these are the small notes that appear to describe methods and properties when you hover your mouse over them. If you wish to make use of these summary notes, make sure the DataCore.xml exists wherever you place the DataCore.dll (before your reference it into your project)


Note: DataCore v3 targets .NET 4.5.1
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Open your .NET Solution (.sln) aka your VS project and add the DataCore.dll as a reference
You can do this by right clicking References in the solution explorer and clicking 'Browse'
then navigating to where you have choosen to store the DataCore.dll (probably should store it beside your launcher/tool)
Now simply add: using DataCore; at the top of each of your .cs that will need it's functionality.

///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////


Current Change-Log
  • Secondary Save() overload removed
  • Primary Save() arguments clarified
    • Arguments: (ref List<IndexEntry> index, string buildDirectory, bool isBlankIndex, bool encodeNames)
    • buildDirectory example: C:\Rappelz\
    • isBlankIndex = true will save a data.blk (blk [blank] functionality not implemented yet)
  • Internal bool NamesHashed added to Core.cs (tracks if caller loads data.000 with names hashed/unhashed (not for callers use)
  • GetEntry(ref List<IndexEntry> index, string inName) now accepts Hashed/Unhashed names
  • GetEntry(ref List<IndexEntry> index, string inName) summary text updated accordingly
  • GetFileEntry removed
  • New methods to generate file hashes (MD5/SHA512) added
    • GetFileMD5(ref List<IndexEntry> index, string dataDirectory, string fileName)
    • GetFileMD5(string dataDirectory, int dataId, long offset, int length, string fileExt)
    • GetFileSHA512(ref List<IndexEntry> index, string dataDirectory, string fileName)
    • GetFileSHA512(string dataDirectory, int dataId, long offset, int length, string fileExt)

Bug Reporting

Just drop a post in this thread (try to be as descriptive as possible) or send me a PM here at the boards.

Viruses

I am a trusted developer here and I don't feel the need to include VirusTotals etc.. in my work, if you do not trust it, it is simple, do not use it.

Special Thanks
  • xXExiledXx aKa Dustin -- Without you I would never have even began coding in c#, I would have never had a good reference in DataBurner and I would not be the developer I am today, thanks man!
  • Glandu2 -- Without you I would still not understand several core concepts in Data Management and I would have never conceived the great ways we use to manage data flow. Also for testing!
  • The Haters -- Without all of you I would have just been another guy making lua scripts and pretending to be a developer, thanks for always pushing me!
  • Adi -- For testing
  • Fyben -- For testing and being my nyukka




SilentWisdom is offline  
Thanks
12 Users
Old   #2
 
elite*gold: 70
Join Date: Dec 2012
Posts: 276
Received Thanks: 155
Good job . Can I add to the Download center?


.BlackCat. is offline  
Old   #3
 
elite*gold: 0
Join Date: Jul 2015
Posts: 206
Received Thanks: 223
ofc
SilentWisdom is offline  
Thanks
1 User
Old   #4
 
elite*gold: 0
Join Date: Nov 2011
Posts: 980
Received Thanks: 1,007
Nicely done. Hopefully you will wake up too.


TheOnlyOneRaskim is offline  
Old   #5
 
elite*gold: 0
Join Date: Oct 2012
Posts: 124
Received Thanks: 12
Quote:
Originally Posted by SilentWisdom View Post
  • The Haters -- Without all of you I would have just been another guy making lua scripts and pretending to be a developer, thanks for always pushing me!
LOL then i must find more haters, so you can make more tools like this
MrStubborn is offline  
Thanks
2 Users
Old   #6
 
elite*gold: 0
Join Date: Jul 2015
Posts: 206
Received Thanks: 223
Minor Updates

Announcing DataCore v3.0.0.3
  • Load overload: bool decodeNames added
    • false is a performance increase for programs that do not need to display decoded names to the user
    • true is a performance loss but allows for displaying decoded names to the user
  • File Method: GetFileBytes(string dataDirectory, int dataId, long offset, int length) added
    • Returns the collection of bytes that represents a file in the data.*** storage system. (handy for doing hash calculations on the files byte[])

WARNING!!! IMPORTANT!!!

If you have previously included DataCore in an application be mindful that your core.Load(string filePath) will thrown an error as it will need the new overload! (e.g. core.Load(string filePath, bool decodeNames)
SilentWisdom is offline  
Thanks
1 User
Old   #7
 
elite*gold: 0
Join Date: Jul 2015
Posts: 206
Received Thanks: 223
v3.0.0.3 Bug Fixes Released
  • Save overload: bool encodeNames added
  • Updated functionality of method: UpdateFileEntry()
    • Originally intended to accept decoded fileName as part of string filePath
    • Updated to accept encoded fileName as part of string filePath
  • Added proper encoding to method: GetFileBytes()

WARNING!!! IMPORTANT!!!

If you have previously included DataCore in an application be mindful that your core.Save(ref index, string filePath) will thrown an error as it will need the new overload! (e.g. core.Save(ref index, string filePath, bool encodeNames)
SilentWisdom is offline  
Old   #8
 
elite*gold: 0
Join Date: Jul 2015
Posts: 206
Received Thanks: 223
Announcing DataCore v3.0.0.4
  • Core.Import/UpdateFileEntry methods switched from OnTotal* events to OnCurrent*
  • Core.UpdateFileEntry() overload added with argument (ref List blankIndex) to update files while tracking orphaned space
  • Restructured code in Import/UpdateFileEntry methods
  • Core.Create() method now reports a totalProgress/totalStatus
  • Core.Load() method now reports a TotalStatus message and CurrentProgress values
  • Core.Load() reportInterval (int) argument added, controls amount of bytes to process before reporting position to caller (progress reporting)
  • Core.Load() overloads for data.000 and data.blk merged
  • Structures.EventArgs.CurrentEventArgs changed to long from int
  • IndexEntry class moved from DataCore namepsace to DataCore.Structures
  • BlankIndexEntry and IndexEntry merged
  • Core.Save() now reports progress via OnTotal/OnCurrent events
  • Core.Save() changed from bool to void (bool always returned true)
  • Core.Save() (all overloads) code optimized
  • Core.Save() overload added to properly save .blk index as a .000 would be
  • Added new Core.GetEntriesByDataId() overload to search BlankIndex
  • Added Core.GetClosestEntry(ref List index, int dataId, int minSize, int maxSize) method to get the closest matching IndexEntry from the data.blk

WARNING!!! The Load/Save arguments have changed since the last release! Make sure to update your project calls to these methods accordingly

This also goes for including the IndexEntry structure into your class, as noted above you can no longer call it by: using DataCore; but instead by adding a new using: using DataCore.Structures;
SilentWisdom is offline  
Thanks
2 Users
Old   #9
 
elite*gold: 0
Join Date: Sep 2013
Posts: 22
Received Thanks: 0
Great tool!

I am currently working on a program to use the DataCore and I found some bugs:
First and most annoying: while the core runs any function the program doesn't respond.
This leads into problem #2: the progress-Events won't be called.
(This means, that the window can not get focus back - if lost - the button inside the program cant be clicked/focused/whatever else you can do with buttons, any ToolTips won't appear, and progressbars - controlled by the progress-Events - won't change - until the core finished working on his function and is ready again - for example dumping files)

The third bug is that the core.Save-Method always throws a IO.PathNotFound-Exception even if the given path exists.

Dumping, exporting files and building the data.001-8 Files work fine. (I havn't worked with deleting and updating enough to say anything about these functions but as far as I am they are working fine too )

Again big THANKS for the great work!


(I apologize for any grammar/spelling mistakes, but I am not a native speaker)
Rappler007 is offline  
Old   #10
 
elite*gold: 0
Join Date: Jul 2015
Posts: 206
Received Thanks: 223
Quote:
Originally Posted by Rappler007 View Post
Great tool!

I am currently working on a program to use the DataCore and I found some bugs:
First and most annoying: while the core runs any function the program doesn't respond.

This leads into problem #2: the progress-Events won't be called.
(This means, that the window can not get focus back - if lost - the button inside the program cant be clicked/focused/whatever else you can do with buttons, any ToolTips won't appear, and progressbars - controlled by the progress-Events - won't change - until the core finished working on his function and is ready again - for example dumping files)

Firstly, you must understand: The DataCore is not multi-threaded by nature this is assumed to be handled inside of your programs code. I suggest using the TASK API.

Example: Task.Run(() => { core.Load(); });

P.S.

Running the function inside a task is the same as using a backgroundworker or thread. If you attempt to directly report event information to a control create on the program thread you will get a "Cross-Thread Operation" exception. So when you write your reporting events do it like so:

Example: core.ErrorOccured += (o,x) => { this.Invoke(new MethodInvoker(delegate {errorLabel.Text = x.Error; }))};

this.Invoke can only be used from the base thread the program is started on!


The third bug is that the core.Save-Method always throws a IO.PathNotFound-Exception even if the given path exists.

I guess I left the naming convention here for the path argument a bit ambiguous. The argument: buildPath expects the whole path like (C:/Rappelz/data.000)

Dumping, exporting files and building the data.001-8 Files work fine. (I havn't worked with deleting and updating enough to say anything about these functions but as far as I am they are working fine too )

Again big THANKS for the great work!


(I apologize for any grammar/spelling mistakes, but I am not a native speaker)
Thanks for your concerns if there is anything else I can do to help please let me know.

P.S.

v3.0.0.4 Bug Fixes and Minor Updates Release
  • Secondary Save() overload removed
  • Primary Save() arguments clarified
    • Arguments: (ref List<IndexEntry> index, string buildDirectory, bool isBlankIndex, bool encodeNames)
    • buildDirectory example: C:\Rappelz\
    • isBlankIndex = true will save a data.blk (blk [blank] functionality not implemented yet)
  • Internal bool NamesHashed added to Core.cs (tracks if caller loads data.000 with names hashed/unhashed (not for callers use)
  • GetEntry(ref List<IndexEntry> index, string inName) now accepts Hashed/Unhashed names
  • GetEntry(ref List<IndexEntry> index, string inName) summary text updated accordingly
  • GetFileEntry removed
  • New methods to generate file hashes (MD5/SHA512) added
    • GetFileMD5(ref List<IndexEntry> index, string dataDirectory, string fileName)
    • GetFileMD5(string dataDirectory, int dataId, long offset, int length, string fileExt)
    • GetFileSHA512(ref List<IndexEntry> index, string dataDirectory, string fileName)
    • GetFileSHA512(string dataDirectory, int dataId, long offset, int length, string fileExt)

Binaries will no longer be released here! Download them from the github in the OP
SilentWisdom is offline  
Thanks
2 Users
Old   #11
 
elite*gold: 0
Join Date: Dec 2014
Posts: 8
Received Thanks: 0
can you give me DataCore.dll please
[ProGramer]sFa7 is offline  
Old   #12
 
elite*gold: 0
Join Date: Mar 2016
Posts: 38
Received Thanks: 13
Quote:
Originally Posted by [ProGramer]sFa7 View Post
can you give me DataCore.dll please
Dark Blaze is offline  
Old   #13
 
elite*gold: 0
Join Date: Jul 2015
Posts: 206
Received Thanks: 223
It is best to use releases from the github or to download the repo and compile.


SilentWisdom is offline  
Reply



« Previous Thread | Next Thread »



All times are GMT +2. The time now is 16:06.


Powered by vBulletin®
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
SEO by vBSEO ©2011, Crawlability, Inc.

Support | Contact Us | FAQ | Advertising | Privacy Policy
Copyright ©2017 elitepvpers All Rights Reserved.