How to may a REALLY big array?

This is a discussion on How to may a REALLY big array? within the C Programming forums, part of the General Programming Boards category; Hi,
I want to make a BIG array with about 10M elements, but it seems like I'm overflowing the stack, ...

the I save a lot of data in bd, but the seg fault comes before that. tried valgrind and it points to where I initialize variables the lines under I defined bd, so I think I'm overflowing the stack ... i've also try to allocate memory like this:

You are doing it correct. You're just using too much of it.
You can increase the stack size, make it global or avoid it by using lesser pointers. You could also, for example, group several pointers into a struct and allocate it on the heap, thus reducing the amount of pointers you use.
Or, you could allocate your big pointer array on the heap and access it with another pointer, so it becomes pointer-to-pointer, see mats's example below.

You are doing it correct. You're just using too much of it.
You can increase the stack size, make it global or avoid it by using lesser pointers. You can, for example, group several pointers into a struct and allocate it on the heap, thus reducing the amount of pointers you use.

Ok, thanks you for your answers, but I'm kind of new on the memory management in C, so how do I increase the stack size? how do I allocate memory on the heap?

Don't malloc use the heap? Are the pointers at the heap but the array of pointers at the stack which I'm overflowing? But how do I put the array of pointers at the heap? and how do I increase the size of the heap if I need to?

Ok, thanks you for your answers, but I'm kind of new on the memory management in C, so how do I increase the stack size?

This is compiler and platform specific, so it should be avoided if possible.

Don't malloc use the heap?

Yes, it does.

Are the pointers at the heap but the array of pointers at the stack which I'm overflowing?

The stack is overflowing with a lot of pointers on it.

But how do I put the array of pointers at the heap?

See mats's example.

and how do I increase the size of the heap if I need to?

You don't. The heap is typically limited only to the amount of virtual memory left (which could be around 1.8 GB or so-ish).
Let's not get too complex. It's not 100% how it works, but it's close enough. Don't gobble up unnecessary memory though.

The code I posted allocates the array on the heap. And yes, malloc() and it's close relatives give you space on the heap. Since the heap can easily be 100-1000 times larger than the stack, you should be OK.

You should, however check if bd == NULL or bd[i] == NULL after the respective calls to malloc, and if so print something and exit the program, as this indicates that there wasn't enough memory.

Increasing the stack is something you need to tell the compiler as part of teh building of the application.

The code I posted allocates the array on the heap. And yes, malloc() and it's close relatives give you space on the heap. Since the heap can easily be 100-1000 times larger than the stack, you should be OK.

You should, however check if bd == NULL or bd[i] == NULL after the respective calls to malloc, and if so print something and exit the program, as this indicates that there wasn't enough memory.

Increasing the stack is something you need to tell the compiler as part of teh building of the application.

--
Mats

Ahh, ok thanks will try your code, must have been writing my post same time as you did. just saw it.