[C++]Binary files

03/05/2010 09:59 Nullable#1

Okay, I'm completely stuck loading a struct from a binary file, saving looks fine to me but on loading i get a bad_allocation exception, may someone please show me how to load/save a struct like this(yes i searched for too many hours and 0 results)

Code:

	typedef struct CColumn
	{
		int ID;
		string UniqueName;
	}Column;
	typedef struct CVar
	{
		string ColName;
		string Value;
	}Variable;
	typedef struct CRow
	{
		int Number;
		Variable * Variables;
	}Row;
	typedef struct CTable
	{
		int ID;
		int RowsCount;
		int ColumnsCount;
		string Name;
		Column * Columns;
		Row * Rows;
	}Table;

Table struct has to be saved in a file, then loaded from the file; Columns and Rows are dynamic arrays.

If the load/save functions are needed please tell me and I'll post them.
Mfg,
Nullable.

03/05/2010 11:11 schlurmann#2

You know your loading function is not working and you need help, but you don't post it? Yea I thought so...

03/05/2010 11:14 Nullable#3

Quote:

Originally Posted by schlurmann

You know your loading function is not working and you need help, but you don't post it? Yea I thought so...

My loading function is full of useless commented out tests, basically i'd say crap, that's why i'd rather have an example than post my crappy method :)
anyhow:

Code:

	void CDatabase::LoadTable(string Name)
	{
		try
		{
			fstream TableStream;
			TableStream.open(Name.c_str(), ios::in | ios::binary);
			Table * T = new Table();
			char * Header = new char[sizeof(int)];
			TableStream.read(Header, sizeof(int));
			TableStream.seekg(sizeof(int), ios::beg);
			BufferBuilder bHeader((byte*)Header, sizeof(int), true);
			int TableSize = (int)bHeader.ReadUInt32();
			cout << TableSize << endl;
			/*char * Stream = new char[TableSize];
			TableStream.read(Stream, TableSize)
			BufferBuilder bStream((byte*)Stream, TableSize, true);
			T->ID = (int)bStream.ReadUInt32();
			unsigned char stSize = bStream.ReadByte();
			T->Name = bStream.ReadString(stSize);
			T->ColumnsCount = (int)bStream.ReadUInt32();
			T->Columns = new Column[T->ColumnsCount];
			for(int i = 0; i < T->ColumnsCount; i++)
			{
				T->Columns[i].ID = (int)bStream.ReadUInt32();
				unsigned char StrSize = bStream.ReadByte();
				T->Columns[i].UniqueName = bStream.ReadString(StrSize);
			}
			T->RowsCount = bStream.ReadUInt32();
			T->Rows = new Row[T->RowsCount];
			for(int i = 0; i < T->RowsCount; i++)
			{
				T->Rows[i].Number = bStream.ReadUInt32();
				T->Rows[i].Variables = new Variable[T->ColumnsCount];
				for(int i2 = 0; i2 < T->ColumnsCount; i2++)
				{
					unsigned char ColSize, ValSize;
					ColSize = bStream.ReadByte();
					T->Rows[i].Variables[i2].ColName = bStream.ReadString(ColSize);
					ValSize = bStream.ReadByte();
					T->Rows[i].Variables[i2].Value = bStream.ReadString(ValSize);
				}
			}
			this->Tables.insert(make_pair<int, Table*>(T->ID, T));*/
			TableStream.close();
		}
		catch(exception& exp)
		{
			printf(exp.what());
		}

Problems start when reading the header(supposed to return 92) but it returns a huge -ve integer(-12283453 not exact).
so

Quote:

char * Stream = new char[TableSize];

this part would return a bad_allocation exp(-ve capacity ftw!) hence i can load nothing
I also tried writing the struct itself into the file and load it same way, but it never works(also returns bad_allocation)
BufferBuilder is just a small class to write data to the buffer..

03/05/2010 12:13 xNopex#4

How about using C++-Strings instead of using old C-strings in your C++ Project? By using the std::string class you will not have problems like a buffer overflow or allocating memory errors. Furthermore you should de-allocate the memory you have allocated with "new" otherwise you will get memory leaks especially when you use pointers.

03/05/2010 16:23 Nullable#5

Quote:

Originally Posted by xNopex

How about using C++-Strings instead of using old C-strings in your C++ Project? By using the std::string class you will not have problems like a buffer overflow or allocating memory errors. Furthermore you should de-allocate the memory you have allocated with "new" otherwise you will get memory leaks especially when you use pointers.

I didn't mention my bufferbuilder class already does the deleting in the destructor, and about purely using the string class.. I don't think that even relates to the problem(but i might actually try doing it, thanks for the idea), because i never get the size header in the file correctly, i did some tests and the buffer written is fine, contains the header, and everything in the table struct, i even loaded it after writing and it looks all fine, but as soon as it is written to the file, somehow i can't seem to load the size header correctly, which then hinders me from completing loading(I can't get any further, don't know the correct size).

03/05/2010 18:14 xNopex#6

Mhm. Maybe it would be a good idea if you show us the saving function. But why do you use a binary file? It would be easier if you save it like this:

Save.sav:

Code:

{CTABLE}
[ID] = 12345
[RowsCount] = 12
[ColumnsCount] = 24
[Name] = Test
{CCOLUMN}
[ID] = 11
[UniqueName] = Bla
{CROW}
[Number] = 1
{CVAR}
[ColName] = Blub
[Value] = val

Now you can easily read in a line by using getline(). Then you have only to parse the string. For example:

Code:

#include <iostream>
#include <string>
#include <vector>
#include <sstream>
#include <fstream>


using namespace std;

typedef struct CColumn
{
    int ID;
    string UniqueName;
}Column;

typedef struct CVar
{
    string ColName;
    string Value;
}Variable;

typedef struct CRow
{
    int Number;
    Variable * Variables;
}Row;

typedef struct CTable
{
    int ID;
    int RowsCount;
    int ColumnsCount;
    string Name;
    Column * Columns;
    Row * Rows;
}Table;

void LoadFile( string* file, Column* col, Variable* var, Row* row, Table* tab )
{
    istringstream isst;
    vector<string> file_content( 0 );
    int rTable_ID, rColumn_ID, rTable_Rows, rTable_Columns, rRow_Number;
    string rTable_Name, rColumn_Name, rVar_Name, rVar_Value, temp;

    ifstream in( file->c_str() );
    while( ! in.eof() )
    {
        getline( in, temp );
        file_content.push_back( temp );
    }
    in.close();

    temp = file_content.at( 1 ).substr( 7 );
    isst.str( temp );
    isst >> rTable_ID;
    temp = file_content.at( 2 ).substr( 14 );
    isst.clear();
    isst.str( temp );
    isst >> rTable_Rows;
    temp = file_content.at( 3 ).substr( 17 );
    isst.clear();
    isst.str( temp );
    isst >> rTable_Columns;
    rTable_Name = file_content.at( 4 ).substr( 9 );
    temp = file_content.at( 6 ).substr( 7 );
    isst.clear();
    isst.str( temp );
    isst >> rColumn_ID;
    rColumn_Name = file_content.at( 7 ).substr( 15 );
    temp = file_content.at( 9 ).substr( 11 );
    isst.clear();
    isst.str( temp );
    isst >> rRow_Number;
    rVar_Name = file_content.at( 11 ).substr( 12 );
    rVar_Value = file_content.at( 12 ).substr( 10 );

    col->ID = rColumn_ID;
    col->UniqueName = rColumn_Name;
    var->ColName = rVar_Name;
    var->Value = rVar_Value;
    row->Number = rRow_Number;
    row->Variables = var;
    tab->ID = rTable_ID;
    tab->RowsCount = rTable_Rows;
    tab->ColumnsCount = rTable_Columns;
    tab->Name = rTable_Name;
    tab->Columns = col;
    tab->Rows = row;
}

int main()
{
    Column col;
    Variable var;
    Row row;
    Table tab;

    string file = "save.sav";
    LoadFile( &file, &col, &var, &row, &tab );
    cout << "Table ID: " << tab.ID << "\nTable RowsCount: " << tab.RowsCount << "\n";
    cout << "Table Columns Count: " << tab.ColumnsCount << "\nTable Name: " << tab.Name << "\n";
    cout << "Table Columns: " << tab.Columns->ID << " " << tab.Columns->UniqueName << "\n";
    cout << "Table Rows: " << tab.Rows->Number << " " << tab.Rows->Variables->ColName << " " << tab.Rows->Variables->Value;
    cin.get();
    return 0;
}

03/05/2010 18:15 ImmuneOne#7

Quote:

Originally Posted by xNopex

How about using C++-Strings instead of using old C-strings in your C++ Project? By using the std::string class you will not have problems like a buffer overflow or allocating memory errors. Furthermore you should de-allocate the memory you have allocated with "new" otherwise you will get memory leaks especially when you use pointers.

Self-explanatory
Stupid

You should of have noticed that the OP is using byte buffers instead of strings.

03/05/2010 18:24 Nullable#8

Quote:

Originally Posted by xNopex

Mhm. Maybe it would be a good idea if you show us the saving function. But why do you use a binary file? It would be easier if you save it like this:

Ehm, Table contains dynamic arrays of Column, and Row, then Row contains a dynamic array of Variables, my idea is to dump them all into a file from the table struct, and then be able to read them all dynamically in the table struct with their dynamic arrays initialized and filled up, without even specifying the number of variables that exist in the table(supposed that i don't know how many, but i would know when loading from the file) but i might tweak your method a bit to satisfy the needs, thanks.

Edit:
Problem solved, thanks.

03/06/2010 02:04 flo8464#9

Quote:

Now you can easily read in a line by using getline().

getline() is completly wrong when working with binary files.

Quote:

bad_allocation

How many bytes are you reading? This exception is normally thrown if no suitable pages can be allocated.

03/06/2010 08:28 xNopex#10

I know that getline() is completely wrong when working with binary files. That's why I suggested to save the structure like a ".ini" file. You should read my posts better :confused:

03/06/2010 10:23 flo8464#11

Quote:

Originally Posted by xNopex

I know that getline() is completely wrong when working with binary files. That's why I suggested to save the structure like a ".ini" file. You should read my posts better :confused:

Yeah, ands thats a waste of space and much less efficient. ;)

03/06/2010 11:02 xNopex#12

Oh come on. Any time I try to help, you criticise my suggestions. First ImmuneOne complained about my "stupid" idea but he did not have a better suggestion instead. Then you moan about my idea to use a filesystem like an ini-file. Of course he can only save the needen information:

Code:

12345|12|24|Test|11|Bla|1|Blub|val

Furthermore he can create a database system to handle the dynamic arrays of the structures.

03/06/2010 11:41 flo8464#13

Yeah, but that requires parsing numbers etc ;)
Try that with a file ~1GB or bigger. It will take eternities.

And well, as far I understood the author, his problem is that he allocates to much memory.

03/06/2010 11:53 xNopex#14

You will not use binary or ini - files to save information with a size of ~1GB. For this you usually use a database.

Quote:

And well, as far I understood the author, his problem is that he allocates to much memory.

As far as I have understood the author, his problem is solved:

Quote:

Edit:
Problem solved, thanks.

03/06/2010 14:52 Nullable#15

I didn't really mention how was my problem solved, but anyway it was a reason different from anything posted all over the latter posts *cough* wrong path *cough*