RDB content vs ARCADIA

sadkingbilly · 03/08/2013, 23:58

Hello

I'm in the process of trying to be able to extract the data from the RDB files, and I use what's in arcadia to do so.

However, there seem to be some tiny differences between the content of each of these: there are some things in the database that are not in the client's RDB files.

For instance, I haven't been able to find the associated rdb for tables JobLevelBonus and CreatureLevelBonus. (Using 7.4 server files)

Have I missed something? If not, how do ppl fill the missing data in the DB with a new game version?

Also, I've tried to search everywhere (in the RDBs and other client files I could reverse engineer) and couldn't figure out if the mobs had their location specified in the clients files. I know they're in a lua script on the server, but I came to the conclusion that it was the only place they were. Am I right?

Thanks

Xijezu · 03/09/2013, 00:20

Quote:

Originally Posted by sadkingbilly

For instance, I haven't been able to find the associated rdb for tables JobLevelBonus and CreatureLevelBonus. (Using 7.4 server files)

Have I missed something? If not, how do ppl fill the missing data in the DB with a new game version?

Some of the tables don't have a RDB-file, e.g. MonsterSkillResource. Also some of them have a different name than the table, e.g. SummonResource would be db_creature.rdb, StateResource would be db_tenacity.rdb, etc.

Quote:

Originally Posted by sadkingbilly

Also, I've tried to search everywhere (in the RDBs and other client files I could reverse engineer) and couldn't figure out if the mobs had their location specified in the clients files. I know they're in a lua script on the server, but I came to the conclusion that it was the only place they were. Am I right?
Thanks

Lua & Mapfiles are containing the Mobspawns, but you can also find some in the Database (RoamingPoint, InstanceDungeonMonsterRespawnResource, HuntaholicMonsterRespawnResource).

sadkingbilly · 03/09/2013, 04:17

Thanks for your input!

So that confirms what I thought, not everything is in the client; I also figured that the names were not directly the same from tables to RDBs when I tried to find the stats.

What I'm wondering is: how do ppl go to have proper values when there is an update (from 7.4 to 8.1, for instance) which introduces new content? If the content is not available in the RDBs how do they get the right data to fill the new version's database (8.1 creature level bonus for new pets for instance)?

Thanks again!

Strange2010 · 03/09/2013, 07:25

Quote:

Originally Posted by sadkingbilly

Thanks for your input!

So that confirms what I thought, not everything is in the client; I also figured that the names were not directly the same from tables to RDBs when I tried to find the stats.

What I'm wondering is: how do ppl go to have proper values when there is an update (from 7.4 to 8.1, for instance) which introduces new content? If the content is not available in the RDBs how do they get the right data to fill the new version's database (8.1 creature level bonus for new pets for instance)?

Thanks again!

you can't always get new content's data , it is in some cases only and those cases are when there is a new item , creature , lootpet , and everything that you can get from rdb (PS : people get them from kts rdb)
but if there is a new thing that cannot be implemented by just converting rdb to sql , you can get nothing , as an example in 8.1 there is the new arena system and that could never be implemented without a gameserver that can read the new tables in arcadia and the changes in telecaster's procedures or tables .
As for the creature level bonus , i think you can insert them manually depending on previously inserted data .

sadkingbilly · 03/12/2013, 20:10

Kk thanks a lot!

Skamorzo · 03/13/2013, 11:32

It would be great if I can learn from your experience how to translate RDBs into SQL Tables... I have read all the posts but no one is releasing their .strs file structure (a part from few of them)...

All I was able to find is that these SQL Tables are conntected to these .RDBs:

Code:

SummonResource                        -> db_creature.rdb;
StateResource                         -> db_tenacity.rdb;

Ciao,
Skamorzo.

Strange2010 · 03/13/2013, 13:34

Quote:
Originally Posted by Skamorzo
It would be great if I can learn from your experience how to translate RDBs into SQL Tables... I have read all the posts but no one is releasing their .strs file structure (a part from few of them)...

All I was able to find is that these SQL Tables are conntected to these .RDBs:
Code:
SummonResource                        -> db_creature.rdb;
StateResource                         -> db_tenacity.rdb;
Ciao,
Skamorzo.

to find out the structures you just have to compare values in rdb with it's sql table , very easy (use hex workshop)

Skamorzo · 03/13/2013, 23:17

@Strange2010: wouldn't it be nice to collect all of them? Have a look at my post...

to see what I mean.

As Xijezu stated: Some of the tables don't have a RDB-file, e.g. MonsterSkillResource. Also some of them have a different names. And some others have more fields than their respective SQL Tables.

I have coded a php <-> rdb interface (compatible with structorian) just to help me out in finding the right values to insert into the right SQL table. The cross-check is the hardest part, to me.

I think that a unified program/interface should:

1. take the rdbs from the client and according to its version:
2. unpack/decrypt them;
3. create the right database table structure;
4. let people play with its content;
5. generate the new rdbs with the altered/added values;
6. encrypt and repack them.

Or just encrypt them and put in the Resource folder.

I know there are lots of tools and programs out there that do part or all the things I mentioned, but:

1. They are mostly outdated;
2. They are partially released;
3. They are too many in numbers.

What about having just one?

That's the main reason why I am interested in this topic, as well as in the

one: I need to speed up this discovery in order to concentrate myself in the coding of this unified PHP WEB/Interface.

Ciao,
Skamorzo.

sadkingbilly · 03/14/2013, 02:33

I generally have 4 programs open:
* Structorian -> test the strs/structure of the rdb
* Notepad++ -> edit strs file (you may not need this if you use the c# version, but I don't like the c# version)
* HxD (hex editor) -> best tool to look at the hex content of the file, can be used to search for stuff, etc. )
* MS Sql -> look a the structure of the database/tables, and fetch the content also.

Make sure your RDBs version match the one of the tables in SQL server.

To find the proper table/RDB combination, well often the names are the same, as previously said, but sometimes they're not, as also previously said. You have to figure out. Sometimes 'summon' is used in the DB while 'creature' is used in the RDBs when talking about pets. Things like that. You can also compare size of the RDB with the size of tables in the DB.

Now what I do is copy the table fields in the strs, one by line. Then I check the type of those DB fields and update the strs with the proper types (DB int -> strs u32, small int -> u16, tiny int -> u8, decimal 10,2 -> float, decimal 10,3 -> double, varchar 256 -> str len=0x100, etc.)

Then you need basic sql query writing skills to try and find the first set of data you have found in structorian in the database, then from that you can match the fields with their proper position in the strs; they're often the same. If they're not, you label the name of the srts field UNKNOWN until you find out what it is. Then it's back and forth until you find out all you need. Some DB fields are 'encoded' into u32 bit fields in the RDB so it's another pass of going back and forth until you've correctly mapped those fields.

I don't have any strs files right now but I'll try to post a couple of those I have made soon.

Oh, and as a miscellaneous tip, structorian complains if you have the file opened in HxD as read-write, so I make my RDBs read only before playing with them.

I hope this helps a bit

sadkingbilly · 03/15/2013, 05:04

Here is what I have.

StatResource -> db_basicstat_rdb.strs

Spoiler

SummonResource -> db_creature_rdb.strs

Spoiler

ItemResource -> db_item_rdb.strs

Spoiler

Code:

// sadkingbilly with db released by GamePortal group (fouad/musta) tuto by ismoke
[byteorder=littleendian]
struct FILE 
{
  str date [len=8];
  [hidden] str [len=(0x80-8)];
  u32 count1;
  child ENTRY [count=count1];
}


struct ENTRY
{
  u32 id;
  u32 name_id;
  i32 type;
  i32 group;
  i32 class;
  i32 set_id;
  i32 set_part_flag;
  i8 grade;
  i32 rank;
  i32 level;
  i32 enhance;
  i32 socket;
  i32 status_flag;
  i8 job_depth;
  i32 use_min_level;
  i32 use_max_level;
  i32 target_min_level;
  i32 target_max_level;
  i32 range_read;
  float weight;
  i32 price;
  i32 huntaholic_point;
  i32 ethereal_durability;
  i32 endurance;
  i32 wear_type;
  bitfield (2)
  {
  i8 limit_fighter[frombit = 10, tobit = 10];//0000010000011100
  i8 limit_hunter[frombit = 11, tobit = 11]; 
  i8 limit_magician[frombit = 12, tobit = 12]; //0001000000011100
  i8 limit_summoner[frombit = 13, tobit = 13];//0010000000000100
  i8 limit_deva[frombit = 2, tobit = 2];
  i8 limit_asura[frombit = 3, tobit = 3];
  i8 limit_gaia[frombit = 4, tobit = 4];
  }
  
  i16 limits_remainder;
  u32 item_use_flag;
  i32 item_use_flag_COPY;
  i32 material;
  i32 summon_id;
  i32 throw_range;
  i16 base_type_0;
  i16 base_type_1;
  i16 base_type_2;
  i16 base_type_3;
  double opt_var1_0;
  double opt_var1_1;
  double opt_var1_2;
  double opt_var1_3;
  double opt_var2_0;
  double opt_var2_1;
  double opt_var2_2;
  double opt_var2_3;
  i16 opt_type_0;
  i16 opt_type_1;
  i16 opt_type_2;
  i16 opt_type_3;
  double base_var1_0;
  double base_var2_0;
  double base_var1_1;
  double base_var2_1;
  double base_var1_2;
  double base_var2_2;
  double base_var1_3;
  double base_var2_3;
  i32 effect_id;
  i16 enhance_0_id;
  i16 enhance_1_id;
  float enhance_0_01;
  float enhance_0_02;
  float enhance_0_03;
  float enhance_0_04;
  float enhance_1_01;
  float enhance_1_02;
  float enhance_1_03;
  float enhance_1_04;
  i32 skill_id;
  i32 state_id;
  i32 state_level;
  i32 state_time;
  i32 cool_time;
  i16 cool_time_group;
  i32 available_period;
  i16 decrease_type;
  str unk0x3100[len=2];
  i32 model_type_dem;
  i32 model_type_def;
  i32 model_type_asm;
  i32 model_type_asf;
  i32 model_type_gam;
  i32 model_type_gaf;
  i8 deco_model_change;
  str model_00[len=0x100];
  str model_01[len=0x100];
  str model_02[len=0x100];
  str model_03[len=0x100];
  str model_04[len=0x100];
  str model_05[len=0x100];
  str model_06[len=0x100];
  str model_07[len=0x100];
  str model_08[len=0x100];
  str model_09[len=0x100];
  str model_10[len=0x100];
  str model_11[len=0x100];
  str model_12[len=0x100];
  str model_13[len=0x100];
  str model_14[len=0x100];
  str model_15[len=0x100];
  str model_16[len=0x100];
  str model_17[len=0x100];
  str texture_filename[len=0x100];
  str drop_type[len=0x100];
  str unkCat[len=3];
  i32 icon_id;
  str icon_file_name[len=0x100];
  str script_text[len=0x200];
  u32 tooltip_id;
}

WorldLocation -> db_worldlocation(ascii).rdb.strs

Spoiler

Code:

// sadkingbilly with db released by GamePortal group (fouad/musta) tuto by ismoke
alias float decimal10_3;
[byteorder=littleendian]
struct FILE 
{
  str date [len=8];
  [hidden] str [len=(0x80-8)];
  u32 count1;
  child ENTRY [count=count1];
}


struct ENTRY
{
  u32 id;
  u32 stringId;
  u32 x;
  u32 y;
  u32 preloading_distance;
  u32 apply_location_name;
  u32 apply_light;
  u32 apply_bgm;
  u32 location_type;
  u32 fog_application;
  i32 time_id;
  i32 weather_id;
  i32 cloud_ratio;
  u32 weather_change_time;
  u32 weather_ratio;
  u32 sky_start_a;
  u32 sky_start_r;
  u32 sky_start_g;
  u32 sky_start_b;
  u32 sky_mid_a;
  u32 sky_mid_r;
  u32 sky_mid_g;
  u32 sky_mid_b;
  u32 sky_end_a;
  u32 sky_end_r;
  u32 sky_end_g;
  u32 sky_end_b;
  decimal10_3 sky_mid_rate;
  u32 diffuse_a;
  u32 diffuse_r;
  u32 diffuse_g;
  u32 diffuse_b;
  u32 ambient_a;
  u32 ambient_r;
  u32 ambient_g;
  u32 ambient_b;
  u32 specular_a;
  u32 specular_r;
  u32 specular_g;
  u32 specular_b;
  u32 fog_a;
  u32 fog_r;
  u32 fog_g;
  u32 fog_b;
  decimal10_3 v_start;
  decimal10_3 v_end;
  decimal10_3 h_start;
  decimal10_3 h_end;
  u16 fixed_terrain_distance;
  u32 cloud_a;
  u32 cloud_r;
  u32 cloud_g;
  u32 cloud_b;
  decimal10_3 land_specular;
  decimal10_3 character_diffuse;
  decimal10_3 character_ambient;
  str sun_texture_file[len=0x100];
  str moon_texture_file[len=0x100];
  str cloud_texture_file[len=0x100];
  u32 bgm1_ID;
  u32 bgm2_ID;
  u32 bgm3_ID;
  u32 bgm1_ratio;
  u32 bgm2_ratio;
  u32 bgm3_ratio;
  u32 environmental_sound1_ID;
  u32 environmental_sound2_ID;
  u32 environmental_sound3_ID;
  u32 environmental_sound4_ID;
  u32 environmental_sound1_ratio;
  u32 environmental_sound2_ratio;
  u32 environmental_sound3_ratio;
  u32 environmental_sound4_ratio;
  u32 environmental_sound_reverb;
  u32 environmental_sound_volume;
  u8 environmental_sound_equalizer;
  u32 minimap_ani_file_ID;
  u32 worldmap_ani_file_ID;
  u32 zoom_map_type;
  u32 display_positoin_x;
  u32 display_positoin_y;
  u32 zoom_x;
  u32 offset_x;
  u32 offset_y;
  u32 size_x;
  u32 size_y;
  u32 blank_resource_ID;
  u32 blank_a;
  u32 blank_r;
  u32 blank_g;
  u32 blank_b;
  u32 zoom_minimap;
  u32 precipitation_min;
  u32 precipitation_max;
  float aurora_t;
  float aurora_a;
  float aurora_s;
  float aurora_h;
}

Enjoy!

Quote:

Originally Posted by Xijezu

Lua & Mapfiles are containing the Mobspawns

Xijezu, are you referring to monster_respawn.lua and mXXX_XXX.nfs? Are there other map files?

Xijezu · 03/15/2013, 06:04

Quote:

Originally Posted by sadkingbilly

Xijezu, are you referring to monster_respawn.lua and mXXX_XXX.nfs? Are there other map files?

monster_respawn & on_startup (or something like that).
About the maps... I can't remember exactly which one it was, but I guess you're right, it should be .nfs. And yes, there are other map files, but for mobs just the .nfs is important.

Skamorzo · 03/15/2013, 18:04

sadkingbilly: look at the spoiler to see what I was using instead...

Spoiler

Code:

[filemask="db_item(ascii).rdb"]

struct HEADER {
  str date [len=128];
  u32 records;
  child ENTRY [count=records];
}

struct ENTRY {
  i32 datum_1; // 101301
  i32 datum_2; // 10101301
  i32 datum_3; // 1
  i32 datum_4; // 1
  i32 datum_5; // 101
  i32 datum_6; // 0
  i32 datum_7; // 0
  i8 datum_8; // 0
  i32 datum_9; // 3
  i32 datum_10; // 1
  i32 datum_11; // 0
  i32 datum_12; // 2
  i32 datum_13; // 0
  i8 datum_14; // 0
  i32 datum_15; // 0
  i32 datum_16; // 0
  i32 datum_17; // 0
  i32 datum_18; // 1040187392
  i32 datum_19; // 0
  float datum_20; // -199520
  i32 datum_21; // -1610610311
  i32 datum_22; // 15
  i32 datum_23; // 2013265920
  i32 datum_24; // 0
  i32 datum_25; // 469762048
  i16 datum_26; // 60
  i16 datum_27; // 0
  i32 datum_28; // 65536
  i32 datum_29; // 50397184
  i32 datum_30; // 0
  i32 datum_31; // 0
  i32 datum_32; // 184549376
  i16 datum_33; // 0
  i16 datum_34; // 3584
  i16 datum_35; // 0
  i16 datum_36; // 0
  float datum_37; // 0
  i32 datum_38; // 4219712
  float datum_39; // 0
  i32 datum_40; // 0
  float datum_41; // 0
  i32 datum_42; // 12588032
  float datum_43; // 0
  i32 datum_44; // 0
  float datum_45; // 0
  i32 datum_46; // 4202496
  float datum_47; // 0
  i32 datum_48; // 0
  float datum_49; // 0
  i32 datum_50; // 0
  float datum_51; // 0
  i32 datum_52; // 0
  i16 datum_53; // 0
  i16 datum_54; // 0
  i16 datum_55; // 0
  i16 datum_56; // 0
  double datum_57; // 0
  double datum_58; // 0
  double datum_59; // 0
  double datum_60; // 0
  double datum_61; // 0
  double datum_62; // 0
  double datum_63; // 0
  double datum_64; // 0
  i32 datum_65; // 184549376
  i16 datum_66; // 0
  i16 datum_67; // -26112
  i32 datum_68; // -1707034215
  i32 datum_69; // 859888025
  i32 datum_70; // 4260659
  i32 datum_71; // 4270080
  i32 datum_72; // 0
  i32 datum_73; // 0
  i32 datum_74; // 0
  i32 datum_75; // 0
  float datum_76; // 0
  float datum_77; // 0
  float datum_78; // 0
  float datum_79; // 0
  i32 datum_80; // 0
  i32 datum_81; // 0
  i32 datum_82; // 0
  char datum_83; // 1
  i32 datum_84; // 0
  i32 datum_85; // 0
  i32 datum_86; // 0
  i32 datum_87; // 0
  i32 datum_88; // 0
  i32 datum_89; // 0
  i16 datum_90; // 0
  str [len=256]datum_91; // sword006
  str [len=256]datum_92; // sword006
  str [len=256]datum_93; // sword006
  str [len=256]datum_94; // sword006
  str [len=256]datum_95; // sword006
  str [len=256]datum_96; // sword006
  str [len=256]datum_97; // sword006
  str [len=256]datum_98; // sword006
  str [len=256]datum_99; // sword006
  str [len=256]datum_100; // sword006
  str [len=256]datum_101; // sword006
  str [len=256]datum_102; // sword006
  str [len=256]datum_103; // sword006
  str [len=256]datum_104; // sword006
  str [len=256]datum_105; // sword006
  str [len=256]datum_106; // sword006
  str [len=256]datum_107; // sword006
  str [len=256]datum_108; // sword006
  str [len=256]datum_109; // 0
  str [len=256]datum_110; // drop_weapon_sword
  char datum_111; // C
  char datum_112; // a
  char datum_113; // t
  str [len=256]datum_114; // icon_weapon_0041
  str [len=512]datum_115; // 
  i32 datum_116; // 0
  i32 datum_117; // 20101301
  if (CurOffset < FileSize) {
    sibling;
  }
}

Thanks for your post, it helped me a lot.

Skamorzo.

sadkingbilly · 03/15/2013, 18:39

All right, thanks!

Quote:

Originally Posted by Skamorzo

sadkingbilly: look at the spoiler to see what I was using instead...

SPOILER

Thanks for your post, it helped me a lot.

Skamorzo.

Yeah, with the names it makes more sense :P

That one was a pain to figure out hehe :P

I used 7.4 DB with 8.1 RDB, so maybe that's why some fields are still unknown, but the data in those don't seem relevant.