[Release] Rappelz RDB Tool with tabs

• Independent tabs
• Support for common mouse click about tabs (middle click on tab close it, double click on empty part of the tab bar create a tab)
• Added a toolbar with common used buttons
• Increased performance when reading/writing files
• Show the search field in red for several secs when the string cannot be found
• Auto load database description DLLs in the same directory as the .exe
• Add manually some other DLL that are not in the .exe directory, like before
• Keep known DLLs when you restart the tool
• Loading a database autoload the current database description DLLs selected in the combobox
• Allow to save SQL options (the password is saved optionally in plain text, if you trust your computer ^^)
• More checks about not saved DB when closing or loading a new DB
• Added information in the statusbar about button & other UI things
• Auto detect the file format when loading a file if not specified
• Support all used charset by Rappelz
• Auto use default database filename or hash / tablename
• Dialog box to convert between hash <=> filename (can also copy files to their converted names in batch)
• Can Read/Write RDB, CSV and SQL tables, can write a SQL script file (but not read)
• Save to existing SQL table without changing it's schema (ie: keep same columns names, order and types). This allow saving a more recent database to an older table
• Implement Database description in LUA instead of C++

• RappelzRDBToolQt.zip or RappelzRDBTool-bin : the tool with all database description DLLs. Require QtDlls4.8.1.zip
• QtDlls4.8.1.zip : If you don't already have it, this is the required runtime DLLs. They have never changed nor they will across versions
• RappelzRDBTool-dev.zip : Needed headers and library to compile a database description DLL
• RappelzRDBTool-db-src.zip : All sources for all database description DLL.
• RappelzRDBTool-db-lua.zip : All LUA script for all database descriptions.

-- This file is a sample Lua script to describe a fake RDB database.

-- All standard libraries are available if needed (io, string, math, ...)
-- The "print" function can be used, its output is redirected to logs.
-- "print" output in the RDB tool GUI is available in the menu "Options -> Show log" with at least "Info" level.

-- Here is a description of the component of a Lua database description file:

-- Required
-- List all fields into the variable rdb.fields. One line with { } per field
-- Each field contains these parameters:
-- - The field name as a string. Example: "id"
--
-- - The field type, visibility flags and additional flags. All flags must be combined with | operator.
--   - The type can be:
--     - TYPE_BIT          : A bit field. The field can be either 1 or 0.
--     - TYPE_INT8         : A 8 bits integer. Values from -128 to 127.
--     - TYPE_INT16        : A 16 bits integer. Values from –32768 to 32767.
--     - TYPE_INT32        : A 32 bits integer. Values from –2147483648 to 2147483647.
--     - TYPE_INT64        : A 64 bits integer. Values from –9223372036854775808 to 9223372036854775807.
--     - TYPE_FLOAT32      : A 32 bits float value.
--     - TYPE_FLOAT64      : A 64 bits float value, also called double.
--     - TYPE_CHAR         : A string field of fixed maximum length. The string is terminated with \0 in the RDB file.
--                           The data length parameter contains the maximum length of the string.
--     - TYPE_VARCHAR_SIZE : A field containing the length of a variable length string of type TYPE_VARCHAR_STR or TYPE_NVARCHAR_STR.
--     - TYPE_VARCHAR_STR  : A string field with variable length. The length in the RDB file is defined by a TYPE_VARCHAR_SIZE field.
--     - TYPE_DECIMAL      : A decimal value with a fixed number of decimals. The data length parameter contains the number of decimals using MAKEINDEXEDVAR. 
--     - TYPE_NVARCHAR_STR : A localized string field with variable length. The length in the RDB file is defined by a TYPE_VARCHAR_SIZE field.
--                           A localized string field use NVARCHAR SQL type instead of VARCHAR.
--
--   - The visibility flag can be:
--     - TYPE_RDBIGNORE    : This flag makes the field not written nor read to or from a RDB file.
--     - TYPE_SQLIGNORE    : This flag makes the field not written nor read to or from a SQL table. If the table is created, it won't contain this column.
--     - TYPE_CSVIGNORE    : This flag makes the field not written nor read to or from a CSV file.
--     - TYPE_GUIIGNORE    : This flag makes the field hidden from the tool GUI interface.
--
--   - Additional flags are:
--     - TYPE_FLAG_KEY     : This flag cause this field to be a primary key when creating a SQL table
--     - TYPE_FLAG_SORT    : This flag add a ORDER BY clause on this field when reading data from a SQL table
--
-- - The field size and/or index
--   - This third parameter is optionnal. When not given, it is set to the default value 1.
--   - If set to a value, the field is an array. For example, a TYPE_INT8 field with size = 3 means an array of 3 8-bits integers.
--   - This parameter can be set using MAKEINDEXEDVAR for variable length string and decimal types:
--     - If the field is of type TYPE_VARCHAR_SIZE, T PE_VARCHAR_STR or TYPE_NVARCHAR_STR:
--       - The first argument of MAKEINDEXEDVAR is the index of the field.
--         - 2 field with the same index and types TYPE_VARCHAR_SIZE and TYPE_VARCHAR_STR/TYPE_NVARCHAR_STR must be defined for variable length string (like in db_string.rdb).
--
--       - The second argument is only used with type TYPE_VARCHAR_STR/TYPE_NVARCHAR_STR.
--         - It contains the maximum string size in the SQL database. This value is used in the SQL type (like varchar(64) for example).
--
--     - If the field is of type TYPE_DECIMAL:
--       - The first argument of MAKEINDEXEDVAR is the number of decimal (after the decimal dot).
--         - MAKEINDEXEDVAR(2, 1) means the field has 2 digits after the decimal dot, like so: 123456.12.
--
--       - The second argument is the field size (and should always be 1 for numbers)
rdb.fields = {
	{"id", TYPE_INT32},                      -- This line describe the first field, it is a 32 bits integer field named "id"
	{"level", TYPE_INT32 | TYPE_RDBIGNORE},  -- This "level" field is a 32 bits integer.
	                                         -- The TYPE_RDBIGNORE flag means this field is not in the RDB format of this file.

	{"price", TYPE_DECIMAL | TYPE_SQLIGNORE | TYPE_CSVIGNORE, MAKEINDEXEDVAR(2, 1)},
	                                         -- The "price" field is a decimal field with 2 decimals (as set by the first argument of MAKEINDEXEDVAR).
	                                         -- The second is the number of decimal values in this field, it should be always 1).
	                                         -- Possible value are 1.00, 504639.12, -165.50
	                                         -- TYPE_SQLIGNORE means this field does not exists in the SQL database.
	                                         -- The TYPE_CSVIGNORE flag means this field is not in the CSV format of this file.

	{"script", TYPE_CHAR | TYPE_CSVIGNORE, 128},   -- The "script" field is a string with a fixed size in the RDB file of 128 bytes.
	                                               -- The TYPE_CSVIGNORE flag means this field is not in the CSV format of this file.

	{"", TYPE_VARCHAR_SIZE, MAKEINDEXEDVAR(0, 1)}, -- This field contains the length of the "name" string field.
	                                               -- This is because the first argument of MAKEINDEXEDVAR of both fields are the same: 0

	{"", TYPE_VARCHAR_SIZE, MAKEINDEXEDVAR(1, 1)}, -- This field contains the length of the "value" string field.
	                                               -- This is because the first argument of MAKEINDEXEDVAR of both fields are the same: 1

	{"name", TYPE_VARCHAR_STR, MAKEINDEXEDVAR(0, 64)},     -- This "name" field is a variable length string with a maximum of 64 characters in the SQL database.
	                                                       -- This string is not localized (and should contains only ASCII) and map to a VARCHAR column in SQL.
	{"value", TYPE_NVARCHAR_STR, MAKEINDEXEDVAR(1, 3999)}, -- This "value" field is a variable length localized string a maximum of 3999 characters in the SQL database.
	                                                       -- This string is localized and can contain characters specific to the used language.
	                                                       -- It maps to a NVARCHAR column in SQL.
}

-- Optional
-- This define the order of the column in the SQL database.
-- It's used when creating the table.
-- It must be a list of columns with \0 at the end of the name.
rdb.sqlColumnOrder = table.concat({
	"id\0",
	"level\0",
	"name\0",
	"script\0",
	"value\0",
	"price\0",
})

-- Optional
-- This define the order of the column in the CSV format
-- It's used when creating and reading a CSV file
-- It must be a list of columns with \0 at the end of the name.
rdb.csvColumnOrder = table.concat({
	"id\0",
	"name\0",
	"level\0",
	"script\0",
	"value\0",
	"price\0",
})

-- Optionnal
-- This override the default RDB file name
-- The default is the name of this file without "xxDatabase.lua" and prepended with "db_"
-- For this exact file, it would be "db_sample".
rdb.defaultFileName = "db_sample"

-- Optionnal
-- This override the default SQL table name
-- The default is the name of this file without "xxDatabase.lua" and concat with "Resource"
-- For this exact file, it would be "SampleResource".
rdb.defaultTableName = "SampleResource"

-- Optionnal
-- This define a function called for each rows when reading or writing data to or from a file/table.
-- It can be used to define values for field not available in all formats
-- Arguments are:
--  - dst: The destination format. Can be:
--    - DF_RDB: when reading/writing a RDB file
--    - DF_CSV: when reading/writing a CSV file
--    - DF_SQL: when reading/writing a SQL table
--
--  - mode: the processing made: reading or writing. Possible values are:
--    - DCT_Read: when reading a file or table
--    - DCT_Write: when writing a file or table
--
--  - row: the row data. Contains each defined fields with values read or to be writen.
--    - When mode == DCT_Read, the row contains values just read.
--      - If values are modified, they will be changed as if they were read as-is and they will be the value shown in the GUI and used for subsequent writes.
--    - When mode == DCT_Write, the row contains values that will be written.
--      - If values are modified, they will be changed both in the target file/table and in the GUI.
--    - The row variable must be used like this:
--      - Given the declared field "level", the value for this field can be read/written with `row.level` or `row["level"]`.
--
--  - rowNum: the index of the current row to be processed. First row has index 0.
rdb.convertData = function (dst, mode, row, rowNum)
	-- Set a default value for the "script" field using "name" and "value" fields
	if mode == DCT_Read and dst == DF_CSV then
		if row.name == "script" and row.value ~= "" then
			row.script = row.value .. "()"
		end
	end

	-- Level is not in the RDB file. It's row index begining with 1.
	if mode == DCT_Read and dst == DF_RDB then
		row.level = rowNum + 1
		-- print can be used to debug code. The message will be shown in the "Show logs" menu with level "Info".
		print("Row index: " .. rowNum .. ", level computed: " .. row.level)
	end

	-- Price is only in RDB format and must absolutly be 15.02 for some reason, so set it when writing a RDB file
	if mode == DCT_Write and dst == DF_RDB then
		row.price = 15.02
	end
end

• 2.0.1 : Fixed a bug that prevented saving files.
• 2.1.0 : Fixed various bugs when loading or writing, you must update both the .exe and RpzRdbBase.dll at once (db description dll don't need to be recompiled). I added Item52aDatabase.dll (and 52b) to open db_item.rdb (and db_item(ascii).rdb) of the 5.2 client posted on epvp by xXExiledXx. (the rdb file compatible with the database is db_item(ascii).rdb)
• 2.2.0 : Better locale management (but still not perfect). Fix sql save on non english OS. Show a error message when can't load DB description DLL
• 2.3.0 :
  - Add default file and table names, allow to use hash as default name.
  - Allow to change locale used to display database strings and to load SQL tables (every Rappelz languages supported) For SQL, the locale is used as the target locale to convert string from unicode to 8 bits. For files, the locale is used to choose how to display data.
  - The default file name and table name use by default the DLL description file without Database.dll and any number before that (so Item521Database.dll has a default name of db_item and table name ItemResource).
  - Fix SQL progress bar.
  - Show DB full name in a tooltip on the tab.
  - Fix TYPE_DECIMAL issue when loading from SQL.
• 2.3.1 : Fix SQL loading issues with 0 length varchar
• 2.3.2 : Add hash <-> filename dialog and fix some problems in provided source files (relative #include and ExportDLL.h)
• 2.3.3 : -Wl,--kill-at no more required for mingw. Show error message about wrong column name or index using OutputDebugString. Fix db_string read when a string has a length of 0
• 2.3.6 : ([Only registered and activated users can see links. Click Here To Register...])
  - Support NVARCHAR column (for db_string) with unicode data (it's possible to store a french StringResource in an Arabic database along with the Arabic StringResource without character loss)
  - Save to sql script: the table name is the filename without ".sql"
  - Fixed hash conversion issue as discovered by MrStubborn
  - Removed primary keys as official tables dont use them and sometime rows are duplicated
  - I renamed all database description dll to have the epic number (I put 8.1 for all of them, them can work for older epic's still) (it's just to remember for which epic these dll were done, databases description dll without epic number still work)
  - Always use dot as decimal separator in the GUI, CSV and SQL script
  - The tool should not crash anymore when loading a non-rdb file as rdb (it will give "Invalid argument" in this case)
  - Updated and added some 8.3 rdb struct description dll (about npc and items)
• 2.3.7 : ([Only registered and activated users can see links. Click Here To Register...])
  - By default, the filter is all file (*.*). If a extension is specified with the filter all files, the format to save/open the file is based on the extension, else it is based on the selected filter. If the file does not have a known extension and the filter "all files" is selected, it default to RDB mode (for hashed name as they don't have an extension of csv, tsv, txt or sql)
  - Add translation support
• 2.4.0 : ([Only registered and activated users can see links. Click Here To Register...])
  - Fixed the save file dialogbox
  - Add a file-to-rename list in the hash/name converter dialog box: When clicking rename, all files in this list will be copied with the converted name as the new filename (maybe still buggy for now)
  - Pressing enter in the hash/name converter convert the text where is the cursor to the other type
• 2.4.2 : ([Only registered and activated users can see links. Click Here To Register...])
  - Add missing arena_points column in Item81Database
  - Fix bugs on various database formats
  - Handle NaN and non-normal float values correctly (SQL Server does not support them)
  - Correct default database names
  - In SQL Options dialogbox, the Data source name is now a combobox which list registered DSN in ODBC
• 2.4.3 : ([Only registered and activated users can see links. Click Here To Register...])
  - Add a checkbox when saving to SQL table to reuse it's schema (IE: use TRUNCATE instead of DROP & CREATE to keep the same columns. All columns of the RDB file MUST exist in the target table. This feature will evolve to allow writing 8.3 database to a 8.1 table by writing only already existing columns)
  - Rename arena_points to arena_point in Item8xDatabase (the original column name has no 's')

• 2.5.0: [Only registered and activated users can see links. Click Here To Register...]
  - Fix crash when loading non existant CSV file
  - Fix possible crash when loading CSV for db_string or db_banwords
  - Fix rounding with negative DECIMAL values
  - Show messagebox when an error occur while reading or writing
  - Fix unknown errors that might appear with RDB that does not have the expected format
  - Fix possible crashes when loading LowQualityWater from SQL or CSV
  - Display correct messages when loading from SQL (wrong table columns and non existent table)
  - When reusing the target SQL table, allow to save only a subset of columns (ie: allow saving 8.3 database into 8.1 table by ignoring new columns). It's not possible to save with this feature when there is a column in the target table that is not in the database to save
  - Better handling of COMMIT/ROLLBACK in case of errors when creating or truncating the target SQL table (keep the old one with all it's data)
  - Add new database descriptions: db_aniinfo.rdb, db_decompose.rdb, db_fieldtypematerial.rdb, db_texture.rdb
  - Fix errors when loading from official SQL tables (8.1 official tables from Strange2010's repack)
  - Correct other invalid default names
  - GUI: Correctly remove all selected lines in Db description DLL manage dialog and Hash / Name file renamer dialog
  - GUI: Add alternate colors in tables for easier reading
  - GUI: Add progress bar for file renamer (this feature actually copy files with the converted filename as the new filename and keep original ones)
  - GUI: Don't remove non existent files in file to rename list at startup
  - GUI: When editing the database, don't save invalid values in memory as 0 (keep the old one instead) (like putting a too big number in a 8 bit integer column)
  - GUI: When renaming filenames in Hash / Name dialogbox, select lines in the files to rename list that have been correctly copied with the target name
• 2.5.1: [Only registered and activated users can see links. Click Here To Register...]
  - Add TYPE_FLAG_SORT to sort rows when reading from SQL (used for jobresource and level tables) (xXExiledXx)
  - Fix flags when reading from SQL db
  - Fix random columns for Huntaholic and SummonExp (Amaady)
  - Fix Item522Database for 5.2 client (db_item(ascii).rdb) (xXExiledXx)
  - Add SkillTree52Database
  - When failing at reading a RDB file, leave data in the table to see what's wrong (useful when reading a new RDB)
  - The combobox with all database description DLLs is larger (display 20 items instead of 10)
• 2.5.2: [Only registered and activated users can see links. Click Here To Register...]
  - Fix data changes when saving a database. Saving a RDB several times works now (Laladumdum)
  - Added [Only registered and activated users can see links. Click Here To Register...]
  - Added [Only registered and activated users can see links. Click Here To Register...]
• 2.6.0: [Only registered and activated users can see links. Click Here To Register...]
  - Use SQL server IP & Port directly in SQL config
  - Add possibility to use a connection string in SQL config
  - If an edited cell has not changed (same text as before), the file is not marked as changed
• 2.7.0: [Only registered and activated users can see links. Click Here To Register...]
  - Add log window: logs can be read now (including SQL error messages). Log can be written from database DLL too using ILog.h
  - When generating .SQL file, add "GO" every 100 insert
• 2.8.0: [Only registered and activated users can see links. Click Here To Register...]
  - Rework IRowManipulator: use row->getData* and row->setData* instead of row->getValuePtr
  - More log message in case of error
  - Updated database DLLs with 9.1 Arcadia tables from revolution team. (Several mapping bugs/unknown name in DLL for previous epics were also fixed).
• 2.8.1: [Only registered and activated users can see links. Click Here To Register...]
  - Fix hashed / non hashed detection of files

• 3.0.0: [Only registered and activated users can see links. Click Here To Register...]
  - Major change: Add LUA support for database description files
  - All database descriptions file were rewritten in LUA.
• 3.1.0: [Only registered and activated users can see links. Click Here To Register...]
  - LUA: Fix various differences between lua and c++ DLL
  - Support .ref files
  - Add ItemRef94Database.lua to handle db_item.ref

Quote:

Originally Posted by TheLeaders Nice RELEASE In this RELEASE Item only exist?

Quote:

Originally Posted by Modamer9 I have this error when I want great file *.dll  Spoiler SkillTreeDisplayDatabase.cpp(20): error C2143: syntax error : missing ';' before '__stdcall' SkillTreeDisplayDatabase.cpp(20): error C2182: 'EDATABASEDLL' : illegal use of type 'void' SkillTreeDisplayDatabase.cpp(20): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int warning C4508: 'registerDBStructure' : function should return a value; 'void' return type assumed ========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ========== I modify the Additional Include Directories property in correct directorie ... and some erorr .. and I write le command in VS command prompt "cl.exe -D_CRT_SECURE_NO_WARNINGS -DBUILDING_CLIENT ...." and some error.

#include "DataType.h"
#include "ExportDLL.h"

cl.exe -D_CRT_SECURE_NO_WARNINGS -DBUILDING_DATABASE /DWIN32 /D_WINDOWS /W3 /Zm1000 /EHsc /GR /MD /O2 /Ob2 /D NDEBUG SkillTreeDisplayDatabase.cpp RpzRdbBase.lib /link /DLL /OUT:SkillTreeDisplayDatabase.dll

Quote:

Originally Posted by sona998 Wondering if there are any 5.2 versions for the dlls? I think the later epics changed alot of the columns for skill tables, and thus when it reads in rdb files ripped from 5.2 client, it doesn't reads it properly.

Quote:

Originally Posted by sona998 Wondering if there are any 5.2 versions for the dlls? I think the later epics changed alot of the columns for skill tables, and thus when it reads in rdb files ripped from 5.2 client, it doesn't reads it properly.

Quote:

Originally Posted by M>M you need to make a dll by your self with the struct of the 5.2 skill.rdb or you can use the table design to find the struct

Quote:

Originally Posted by glandu2 Either the DLL is not compatible with this version, the path used use non recognized characters (sorry for no unicode support, I will try to improve that, for now to workaround that problem, use only path ascii (english) letters). If you compiled the DLL yourself, be sure to create the DLL as a 32 bit DLL (and without removing #pragma lines)

Quote:

Originally Posted by sona998 I've been trying to read row data from db_skill.rdb, db_skilltree.rdb for E4/E5 clients. The source you left for this was great, db_skillstage structure was similar so that one has been done. Been reading through your comments and came across skilltreedatabase.cpp. Most structures for the rdb are similar, i.e. header at start, then total row count, then rows of data one after the other. With skilltree(ascii).rdb, it has a unique structure where it starts with the header, then total row count, then 'x' row count of skills for a job id, then x row skills for said job id, then repeat. i.e. it has an extra number before each set of skills for a job. I've worked out the strucutre for skilltree rdb but not sure in how to represent this using the skilltreedatabase.cpp source, hence your comment about some special behaviour. Perhaps you could provide some pointers into setting up this structure using the skilltreedatabase.cpp source? Code: struct HEADER { str date [len=8]; [hidden] str [len=(0x80-8)]; u32 count1; child entry2[count = count1]; } struct entry2 { u32 rowCount; child ENTRY [count = rowCount]; str offset [len = 52*rowCount]; } struct ENTRY { u32 job_id; u32 skill_id; u32 min_skill_lv; u32 max_skill_lv; u32 lv; u32 job_lv; float jp_ratio; u32 need_skill_id_1; u32 need_skill_id_2; u32 need_skill_id_3; u32 need_skill_lv_1; u32 need_skill_lv_2; u32 need_skill_lv_3; } Cheers, sona.

#pragma comment(linker, "/EXPORT:getSpecialCaseID=_getSpecialCaseID@0")
int EDATABASEDLL DLLCALLCONV getSpecialCaseID() {
	return SPECIALCASE_DOUBLEFORRDB;
}