Storing data in file

This is a discussion on Storing data in file within the C Programming forums, part of the General Programming Boards category; Here is the problem: I need a way to store large amounts of data (e.g. tables of doubles) in a ...

Storing data in file

Here is the problem: I need a way to store large amounts of data (e.g. tables of doubles) in a file and to be able to read the data.
I created 2 types of save files.
One is a .txt file
benefits - its readable
The other is a file that uses fwrite() and fread() to copy an array to it.
benefits - 3 times smaller file size, no extra casting to decimal and rounding.

On small arrays, this worked fine, but when I tried these out on a 20,000x5 array I had problems.
The txt version worked ok, but when I load the 2nd type of file into an array(or maybe when I write the file, it writes gibberish), at some row at around 110-120 the data becomes gibberish. I can't find any problems with my code.
My questions:
Is there something wrong with my code?
Is there some better functions for storing data in files?
Is there some standard way that everyone uses to store a lot of data into files?
Thanks a lot for the help!

If your compiler is not complaining about the commented-out calls (once they're uncommented, of course), it's broken. You are passing a 2d array of double (which winds up being a pointer to an array of double) to a function that's expecting a pointer to double. These are not the same thing! Your compiler is required to issue a diagnostic.

The standard ways to store data are the two you've mentioned: either as text (which with floating point can be problematic, as you note) or as binary (which can be slightly painful if you're sharing the data between different systems). For the latter, you typically mandate byte-order and data format (which would certainly mean an IEEE float of some sort), and leave it up to the programmer to figure out how to convert that to the native format. If you're only ever going to read back data on the same system, of course, you can just write it out in its native form.

From my understanding, the name of an array itself is a pointer to its first number, and it is fine to put it into a function that wants a pointer.
I tried doing:
double *arr = array;
or
double *arr = &(array[0][0]);
and passing arr to the functions but it runs exactly the same.

From my understanding, the name of an array itself is a pointer to its first number, and it is fine to put it into a function that wants a pointer.
I tried doing:
double *arr = array;
or
double *arr = &(array[0][0]);
and passing arr to the functions but it runs exactly the same.

The name of an array can typically be treated as a pointer to its first member. In your case, the first member is an array. Thus you have a pointer to an array of double, as I said. Which, again, is not the same thing as a pointer to a double.

double *arr = &array[0][0] gets you a pointer to the first element of the first array; you are not allowed to go beyond the end of it. You certainly might be able to, but the code is wrong and a compiler is free to optimize in ways that assume you're not breaking the rules.

Since you have an array of array (which will effectively be a pointer to an array), why don't you just tell your functions to expect one? Why try shoehorning it into another type?

You need to open the file as "wb" ... unless you want it adding carriage returns after 0s.

This was the problem lol :P changing "w" to "wb" fixed it. What exactly does the b mean?

You guys are all saying that when I have an array ary[5][10] then ary is a array of 5 pointers, and each pointer is a pointer to an array of 10 items?
From what they taught us in school, and what I read in books, ary[5][10] is just one giant line of memory with 50 spaces.
ary is a pointer to ary[0][0]
ari[0][1] is *(ari + 1)
ari[1][0] is *(ari + 10) (in this example)
ari[3][4] is *(ari + 3*10 + 4)

This seems to be right from tests I did, and this program works when I copy my data into the array[0][0] and onwards. Just change "w" to "wb" and my code works.

Originally Posted by cas

Since you have an array of array (which will effectively be a pointer to an array), why don't you just tell your functions to expect one? Why try shoehorning it into another type?

I don't want to make my function only work for only one certain size of arrays, with my function, it will work for any array of any size, you just need to pass to the function the pointer to the first number, and how many rows and columns there are(to calculate the amount of numbers there are).

You guys are all saying that when I have an array ary[5][10] then ary is a array of 5 pointers, and each pointer is a pointer to an array of 10 items?
From what they taught us in school, and what I read in books, ary[5][10] is just one giant line of memory with 50 spaces.

What it is and how it is stored are two different things. You are guaranteed that the memory is all in a straight row. But that doesn't mean that ary[5][10] is the same as ary[50]. For instance, ary+1 will point to ary[1][0], not ary[0][1].

This was the problem lol :P changing "w" to "wb" fixed it. What exactly does the b mean?

Ok... one more try... When you open a file in "w" mode the library functions expect lines of text that correspond to strings and will format the file accordingly. That is, it's a text file. When you open in "wb" you are turning the text formatting off and now it will write only what you tell it to write.

You guys are all saying that when I have an array ary[5][10] then ary is a array of 5 pointers, and each pointer is a pointer to an array of 10 items?

It is an array arranged into 5 lines of 10 entries each.

From what they taught us in school, and what I read in books, ary[5][10] is just one giant line of memory with 50 spaces.

Well, it's that too. But only in the sense that it is stored as a continuous block of memory. The order of the elements may or may not be what you think they are...

This seems to be right from tests I did, and this program works when I copy my data into the array[0][0] and onwards. Just change "w" to "wb" and my code works.

On that compiler ... there's nothing guaranteeing other compilers will store/access the data in the same order.

I don't want to make my function only work for only one certain size of arrays, with my function, it will work for any array of any size, you just need to pass to the function the pointer to the first number, and how many rows and columns there are(to calculate the amount of numbers there are).

You would be far smarter to take charge of the order yourself... write to disk in loops --working with ari[i][j]-- that respect the array's dimensions. Then create a complimentary function to read it back from disk in the same manner. Then you will have something that stands a chance of working on future compilers and different machines.