IMPORTANT
Please be sure to download most up to date patch along with application.rar, as the latest patch (2012-05-06) contains a much faster conversion system)
This is an updated and much more flexible data converter than my previous one.
Note though, due to its complete flexibility to the point you could use this for almost anything, it runs much slower than the previous one.
However the previous one is not capable of converting the cq_goods to shop.dat.
Please don't forget to change the config file before using it to match your MySQL settings.
Structure Files
How To:
Convert cq_itemtype to Itemtype.dat
Convert cq_goods to Shop.dat
Table Designs:
cq_itemtype:
cq_goods
Defining Structures:
MainConfig File
Used to define main settings of conversion, such as MySQL login, expected output file size and the source/structure file locations.
Must Contain the Following:
Example File ("ShopConfig.ini"):
Source File
Used to define source fields and their values.
Must Contain the Following (For each field):
Special characters:
'|' = Separates blocks of string, can be used to input source fields into MySQL strings, or to separate a field name and value. Its use depends on the type of data retrieval being used.
Example File ("ShopSource.ini"):
Structure Text
Used to define the structure of the output file.
Must contain at least 1 line of text, at least 1 group and at least 1 field.
Special Characters used:
'@' = Used after a "DataType" definition
'#' = Used after an "ArraySize" definition
'[' & ']' = used to enclose the name of a 'SourceFieldValue'
'/c' = used to get the currently selected value in the source field
'/n' = used to select and get the next value in the source field
'|' = used before defining the 'WriteCount' which is how many times the field/group is written to the file before moving on
',' = used to separate fields within a group
'-' = used to separate groups within a group
'(' & ')' = used to enclose a 'ConditionalStatement' for a conditional group
'^' = used to separate multiple conditional statements/groups
'{' & '}' = used to enclose the fields/group to use when the conditional statement is true
'!=','==' ,'<=','>=' = used to evaluate within a conditional statement
Example File (ShopStructure.txt):
Patches:
Applying Patches:
Download all patches,
Extract in order of date (Patch[YYYY-MM-DD])
Override existing files. - Note if you made any custom changes to structure files, backing up is suggested.
2012-04-20
Files: ConvertData.exe, ShopConfig.ini, ShopSource.ini
ConvertData.exe
Changed how the ConvertData.exe loads the config file. Primarily the "ExpectedFileSize" value now allows for more advanced math than before.
The "ExpectedFileSize" still only calculates left to right without any order of precedence but it now allows for bracketed math. An example is how the Shop.dat size is calculated: 4|+|(ShopCount/c * 28)|+|(NonMallItemCount/c * 4)|+|(MallItemCount/c * 452)
This could not be achieved in the previous version.
Fixed a minor error in the /n method of source fields. (May cause future problems)
Changed the display of the command prompt, updating is also done every second now instead of 200times/second, this gives a very slight performance increase.
Screenshot of new status screen:
ShopConfig.ini
Updated the "ExpectedFileSize" value which will calculate the file size correctly, even if you have duplicate entries.
An example is if you have:
id-itemtype-ownerid
1-190000-1207
2-190000-1207
ShopSource.ini
Included some new fields that were needed for the "ExpectedFileSize" field of the newly updated ShopConfig.ini.
Updated some existing fields, to prevent duplicate entries.
NewFields:
UpdatedFields:
2012-05-06
Files: ConvertData.exe, DataConverterLibrary.DLL, ShopSource.ini, ItemtypeSource.ini
ConvertData.exe
Made it so the application doesn't close immediately after conversion was completed. You must push a button to exit once done, this was implemented since now conversion is over a thousand times faster.
Screenshot of new status screen (Creating a 45MB shop.dat from cq_goods with 101,364 entries in just 9seconds!)
DataConverterLibrary
Added new "Enum" - UpdateFrequency
This determines how often the "AquisitionString" should be called, if "None" it will be called once then all the required data will remain in memory, such types like this will be "ShopCount" or "ShopID".
If "Continuous" it will be called each and every time the field is accessed, this will have a heavy load on the cpu if repeated too much.
If "EachRetrieval" it will be called each time the /n key is used on a field, this will have a moderate load on the cpu if repeated too much.
ShopSource.ini
Added a new field, and updated all existing fields, this should be as optimised as possible for speed.
NewFields:
Updated Fields:
2012-05-11
Files: ConvertData.exe, DataConverterLibrary.DLL, ItemtypeConfig.ini, ItemtypeSource.ini, ItemtypeStructure.txt, ShopConfig.ini, ShopSource.ini, ShopStructure.txt, ItemAdditionConfig.ini, ItemAdditionSource.ini, ItemAdditionStructure.txt
Folders: Output, StructureFiles, SourceFiles
ConvertData.exe
Capable of converting multiple files at once now. Just highlight a group of Config files, click and drag onto the exe. Then wait until all are finished, hit any key.
More threading enabled, making multiple conversions even faster now.
Capable of saving files to preferred file names, as per the config "OutputFileName" value.
Screenshot of new status screen (Creating 3 new files, using ItemAdditionConfig/ItemtypeConfig/ShopConfig)
DataConverterLibrary
Changed the MySQL handling to allow for multiple connections, needed to handle multiple simulataneous file conversions.
ItemAddition Structure Files
Added a conversion structure for cq_itemaddition -> ItemAddition.ini, should work with current table design.
New Folders
Added for file management, updated config files accordingly.
Please be sure to download most up to date patch along with application.rar, as the latest patch (2012-05-06) contains a much faster conversion system)
This is an updated and much more flexible data converter than my previous one.
Note though, due to its complete flexibility to the point you could use this for almost anything, it runs much slower than the previous one.
However the previous one is not capable of converting the cq_goods to shop.dat.
Please don't forget to change the config file before using it to match your MySQL settings.
Structure Files
- ItemAddition
- Itemtype
- LevelExp(Not Encrypted)
- MergeInfo
- Shop
How To:
Convert cq_itemtype to Itemtype.dat
- Follow instructions on cq_itemtype table design. Located below.
- Download Application.rar
- Download ItemtypeStructure.rar
- Extract files from rar files.
- Click and drag the "ItemtypeConfig.ini" into the "ConvertData.exe"
- Wait until the program shuts down, you should see it display how many bytes have been written to the file as it proceeds.
Convert cq_goods to Shop.dat
- Follow instructions on cq_goods table design. Located below.
- Download Application.rar
- Download ShopStructure.rar
- Extract files from rar files.
- Click and drag the "ShopConfig.ini" into the "ConvertData.exe"
- Wait until the program shuts down, you should see it display how many bytes have been written to the file as it proceeds.
Table Designs:
cq_itemtype:
cq_goods
Defining Structures:
MainConfig File
Used to define main settings of conversion, such as MySQL login, expected output file size and the source/structure file locations.
Must Contain the Following:
- Heading "[Main]"
- Field and Value for "MySQLLogin"
- Field and Value for "MySQLPassword"
- Field and Value for "MySQLHost"
- Field and Value for "MySQLDatabase"
- Field and Value for "SourceFileName"
- Field and Value for "StructureText"
- Field and Value for "ExpectedFileSize" (can just be set to 0 if not sure)
Example File ("ShopConfig.ini"):
Source File
Used to define source fields and their values.
Must Contain the Following (For each field):
- Heading "[FieldName]"
- Field and Value for "Type" - Method used for getting data
- Field and Value for "String" - the 'aquisition' string used for aquiring data
Special characters:
'|' = Separates blocks of string, can be used to input source fields into MySQL strings, or to separate a field name and value. Its use depends on the type of data retrieval being used.
Example File ("ShopSource.ini"):
Structure Text
Used to define the structure of the output file.
Must contain at least 1 line of text, at least 1 group and at least 1 field.
Special Characters used:
'@' = Used after a "DataType" definition
'#' = Used after an "ArraySize" definition
'[' & ']' = used to enclose the name of a 'SourceFieldValue'
'/c' = used to get the currently selected value in the source field
'/n' = used to select and get the next value in the source field
'|' = used before defining the 'WriteCount' which is how many times the field/group is written to the file before moving on
',' = used to separate fields within a group
'-' = used to separate groups within a group
'(' & ')' = used to enclose a 'ConditionalStatement' for a conditional group
'^' = used to separate multiple conditional statements/groups
'{' & '}' = used to enclose the fields/group to use when the conditional statement is true
'!=','==' ,'<=','>=' = used to evaluate within a conditional statement
Example File (ShopStructure.txt):
Patches:
Applying Patches:
Download all patches,
Extract in order of date (Patch[YYYY-MM-DD])
Override existing files. - Note if you made any custom changes to structure files, backing up is suggested.
2012-04-20
Files: ConvertData.exe, ShopConfig.ini, ShopSource.ini
ConvertData.exe
Changed how the ConvertData.exe loads the config file. Primarily the "ExpectedFileSize" value now allows for more advanced math than before.
The "ExpectedFileSize" still only calculates left to right without any order of precedence but it now allows for bracketed math. An example is how the Shop.dat size is calculated: 4|+|(ShopCount/c * 28)|+|(NonMallItemCount/c * 4)|+|(MallItemCount/c * 452)
This could not be achieved in the previous version.
Fixed a minor error in the /n method of source fields. (May cause future problems)
Changed the display of the command prompt, updating is also done every second now instead of 200times/second, this gives a very slight performance increase.
Screenshot of new status screen:
ShopConfig.ini
Updated the "ExpectedFileSize" value which will calculate the file size correctly, even if you have duplicate entries.
An example is if you have:
id-itemtype-ownerid
1-190000-1207
2-190000-1207
ShopSource.ini
Included some new fields that were needed for the "ExpectedFileSize" field of the newly updated ShopConfig.ini.
Updated some existing fields, to prevent duplicate entries.
NewFields:
UpdatedFields:
2012-05-06
Files: ConvertData.exe, DataConverterLibrary.DLL, ShopSource.ini, ItemtypeSource.ini
ConvertData.exe
Made it so the application doesn't close immediately after conversion was completed. You must push a button to exit once done, this was implemented since now conversion is over a thousand times faster.
Screenshot of new status screen (Creating a 45MB shop.dat from cq_goods with 101,364 entries in just 9seconds!)
DataConverterLibrary
Added new "Enum" - UpdateFrequency
This determines how often the "AquisitionString" should be called, if "None" it will be called once then all the required data will remain in memory, such types like this will be "ShopCount" or "ShopID".
If "Continuous" it will be called each and every time the field is accessed, this will have a heavy load on the cpu if repeated too much.
If "EachRetrieval" it will be called each time the /n key is used on a field, this will have a moderate load on the cpu if repeated too much.
ShopSource.ini
Added a new field, and updated all existing fields, this should be as optimised as possible for speed.
NewFields:
Updated Fields:
2012-05-11
Files: ConvertData.exe, DataConverterLibrary.DLL, ItemtypeConfig.ini, ItemtypeSource.ini, ItemtypeStructure.txt, ShopConfig.ini, ShopSource.ini, ShopStructure.txt, ItemAdditionConfig.ini, ItemAdditionSource.ini, ItemAdditionStructure.txt
Folders: Output, StructureFiles, SourceFiles
ConvertData.exe
Capable of converting multiple files at once now. Just highlight a group of Config files, click and drag onto the exe. Then wait until all are finished, hit any key.
More threading enabled, making multiple conversions even faster now.
Capable of saving files to preferred file names, as per the config "OutputFileName" value.
Screenshot of new status screen (Creating 3 new files, using ItemAdditionConfig/ItemtypeConfig/ShopConfig)
DataConverterLibrary
Changed the MySQL handling to allow for multiple connections, needed to handle multiple simulataneous file conversions.
ItemAddition Structure Files
Added a conversion structure for cq_itemaddition -> ItemAddition.ini, should work with current table design.
New Folders
Added for file management, updated config files accordingly.
