Need some help with a Database Issue

08/06/2010 19:42 Slayer_Trojan#1
Hi, I would like some help on figuring this problem with my database out. (Server type 5165) First off I would like to explain how I thought it worked. Ok, I thought it worked like something out of Excel or how the guides show you. But as I found out its not. How do I get my database to look somewhat readable, and usable to my understanding. No I'm not a idiot, and yes I have less than what I want in experience to code and solve problems like this. All I am asking for is some help with this issue or someone to explain how it works so I can figure out what I am doing.


Thanks for anyone who replies with something helpful!
Zac
08/06/2010 21:20 Slayer_Trojan#2
Anyone?
08/06/2010 21:31 _tao4229_#3
I'll explain.


IIRC (if I'm wrong I'll delete this and explain what it really is) it's a binary file database. This means everything thats dumped into the database (at least characters from what I remember) as raw data. It's not readable through any editor or anything nice and fancy, it's quite literally a raw dump of data. In order to read/be able to edit it like you might think you'd have to write your own program to read the database yourself. You can use a hex editor to read the raw data, but you have to know where the information is stored in the file (you can tell by reading the source).

Instead of say a text file that stores the number 510 in CPs as "CPs=510", the way the database works is that a certain portion of the file is just the number 510 (not in text, in byte form) and the program reads that portion as CPs. If you open it in notepad you'll probably get tons of random symbols/spaces (thats notepad trying to read the numbers as text), and possibly the players name (as thats actually saved as text).

--- Long version
Files are just dumps of bytes, all of them. The difference is how the bytes are interpreted.

A text file is similar, just every byte is interpreted as an ASCII or Unicode character (a letter or number for example).
In these encodings, certain number values equate to symbols, letters, numbers, etc.
So if the data was like this (in bytes)

(Decimal values - normal number system used)
104 | 101 | 108 | 108 | 111 | 0

(Hexadecimal values for the same data - Look up hexadecimal if you dont know it - http://en.wikipedia.org/wiki/Hexadecimal, just basically a base 16 number system vs base 10 that we use)
0x68 | 0x65 | 0x6C | 0x6C | 0x6F | 0x00

In ASCII, these values would be interpreted as 'h' 'e' 'l' 'l' 'o' and then the number 0 is used to end the string. The ASCII encoding is 1 byte per character, and Unicode is similar, although it uses 2 bytes per character so more characters can be used. For example, since a byte is only capable of 256 different values (0-255 is 256 values guys), 2 bytes together are capable of exponentially more values (65536 values to be exact). This means more symbols can be interpreted, such as latin characters, chinese characters, etc.

You don't really need to worry too much about ASCII vs Unicode for now. If you want, you can google it.


For these 'binary data' files, the data is interpreted literally.
If the data is like this
(Dec)
169 | 1 | 0 | 0

(Hex)
0xA9 | 0x01| 0x00 | 0x00

This could be interpreted as 4 separate bytes that mean 4 separate things, or 2 shorts (2byte values), or 1 int(4bytes).

This could for example be the int X of a character, and when you combine the data its the value 425, or 0x1A9. Right next to it could be the Y value of the character.

If you look at the source, it has something like writer.WriteInt32(Character.X); This literally just dumps the data straight to the stream. It's written and read sequentially, so the first thing written is first in the file. The value of X is then read back into memory from the file steam, and used there.

You can write any data into a binary file, as long as you read it back correctly from the right spot. If you look at the source, you'll notice reading/writing goes in the exact same order of variables.

Strings can also be stored in a binary file, and they're encoded just like a text file would be (either ASCII or Unicode).

Using binary files eliminates the need to parse text into integers/parse text into other values, because you can dump raw memory to them, and read the dump back into memory. This equates to faster loading times over text files, but also makes them much harder for the average person to read (while this could also be a good thing - if you're trying to hide something I guess).


You might understand this a lot more if you understand how computer memory works.
08/06/2010 21:54 Slayer_Trojan#4
Anyway to change that to get it into readable files? Like wiping the database that's in there now and making it like a SQL or a different type of document? Anything possible like that? Or should I just start over or something?
08/06/2010 21:57 _tao4229_#5
You could recode how the database stores/loads information obviously. You could use something more 'readable' like text files (XML, .ini, own format, etc), or use SQL (which has nice pretty editors).

This does involve you removing a part of the source and writing your own code to do the same function differently. Yes code.
08/06/2010 22:12 Slayer_Trojan#6
Any possibility of you helping me figure out or some what of tutoring me how to code that? Because if you could that would be SO AWESOME! Lol, but really if you could that would be cool.
08/06/2010 22:14 pro4never#7
As i said in a pm before, there is a sql based lotf out there, not sure how good it is but it would save some of the coading (better to do yourself).


As stated you could make a prog to read/edit the files but that's a bit of work... the conversion of values is easy enough but editing and structure would be a pain.
08/06/2010 22:20 Slayer_Trojan#8
Well, I understand this but programing isn't my strong point networking is but that's a different story. My database is saved as a SQL file already its just when I open it nothing comes up besides what was in there. Not my characters or accounts nothing. Sorry for sounding like a idiot, but I just need a little bit of help understanding how it works and everything. Thanks for helping me and pointing me in the right direction. Really appreciate the help!
08/06/2010 22:30 pro4never#9
Quote:
Originally Posted by Slayer_Trojan View Post
Well, I understand this but programing isn't my strong point networking is but that's a different story. My database is saved as a SQL file already its just when I open it nothing comes up besides what was in there. Not my characters or accounts nothing. Sorry for sounding like a idiot, but I just need a little bit of help understanding how it works and everything. Thanks for helping me and pointing me in the right direction. Really appreciate the help!
Hmmm? So you've created your own database in sql already and are simply trying to make lotf work with it?

Take a look at Elite-CoEmu, it makes use of a very good sql wrapper created by immune that allows for very easy use of sql on a C# source (you could do it yourself but the other sql systems are rather... lacking)

Or do you mean you have a .sql file? those are simply backups and must be loaded into the sql database. Use a program like navicat so you have some gui functionality and then create a database and execute the .sql file to it. It should then populate all the required databases. Assuming your source is already setup to work with a sql database you simply need to enter your sql username/password/host/database name and you should be good to go.
08/06/2010 22:34 Slayer_Trojan#10
Well, it's in my OldCODB file and it says "database.sql". Me being new to this assumed that that's my database upload it to navicat and I'm ready to go. But it doesn't seem to work like that does it? Or am I wrong and it could be and I'm doing something wrong?