file read to std::string constructor throws

I have a
std::string (char *, std::string::size_t )
constructor failing and I am not sure why.

For my small test file there was no problem, but when using
ifstream::read()
with an unsigned int sz of 1Gb (1073741824)
(I think that I used a pos_type 1st time)
and the resulting char* memblock

I first compare against gcount to confirm that sz isnt bigger.

Have I got an overflow problem max_size() returns 4Gb as expected?

Or is it a problem with the file data?

The main problem that I have is that I am trying to chop a very big file (25Gb) xml into manageable chunks. So I loaded the file in 1Gb
chunks and wrote them straight out using ifstream read() and ofstream write(). So my files might not be in the state that I think they are and might be missing an end of file.

1. Post the code (at least some of it) so we can get a better idea of what's going on.
2. Why don't you use an XML library to parse the code. There are hundreds of XML libraries out there. Such libraries can save you a ton of work and they are usually very efficient.
3. Why are you trying to read the entire file at once? Couldn't you process line by line, and not have to worry about overflows at all?

works for test file but not for big file
where it outputs max size as expected:
4294967296<>1073741824
Then visio crashes on the constructor

2. Why don't you use an XML library to parse the code. There are hundreds of XML libraries out there. Such libraries can save you a ton of work and they are usually very efficient.

3. Why are you trying to read the entire file at once? Couldn't you process line by line, and not have to worry about overflows at all
?

The original file that I want to process is 25Gb this presents problems for a very large amount of programs and templates as the size of an int restricts to 2^32 and hence 4Gb input. Especially on windows many things fail at the 4Gb limit.

I would like to process the file line by line eventually but first I want to simply chop it on a single xml tag. To process the file I was wanting to use an optimised validated find routine such as found in the STL.

I also am not sure how to access just a pointer for the file start of data and then iterate char by char, which is why I wanted to use a container

I will try chopping the original 25Gb file into smaller units ~ 1Mb and then see if that can be processed.

I'm not sure why you are using a binary fstream to read text data. It seems like you are causing yourself some extra pain by doing this. If you are really just parsing xml, why not use a standard ascii fstream? Also, To manage/query file sizes, you should use stat or some similar system call to get the file size.
limit.

I would like to process the file line by line eventually but first I want to simply chop it on a single xml tag.

It's actually very easy to parse line by line. If you use an ascii file stream, you can be confident that each line could be easily converted to a string object for parsing.

I also am not sure how to access just a pointer for the file start of data and then iterate char by char, which is why I wanted to use a container

Again, if you use an ascii ifstream, this is trivial. You can simply use the ifstream::get( char& c ) to extract a single character from the file.

So, I recommend you do this:
1. Use stat to query the size of the file for checks and safety
2. Open the ifstream not as ios::binary but regular old ascii ifstream::in
3. If you want to parse a section of the file at a time by converting it into a stream, use getline()
4. If you want to parse a character at a time, use get()
5. If you are not trying to extract the entire file into a single string, these techniques should work for even large files.

Sorry, I did not make myself clear the comment was supposed to indicate that I had tried using ifstream with in as well.

I'm not sure why you are using a binary fstream to read text data.

The problem that I have is that I do not understand why the string cstr() might be failing. The only thing that I am not sure about is the state ot the original file as I used a brute force approach to chop the 25Gbytes

I don't know stat() and could not find much documentation on it. A c struct for getting file info? I looked at sys/stat.h and I think that it still uses an int_32 so would be unusable with the original 25Gig file.

4. If you want to parse a character at a time, use get()

This might be usable but I am not sure what happens around the 4Gig mark. When I said find the chars directly I meant char * pc; equivalent but even char * this won't cope with the file as not big enough.

Hi. so this is actually a continuation from another question of mine[Here](https://www.daniweb.com/programming/software-development/threads/506795/dynamically-add-values-into-datagridview-cell-from-listbox-vb2010) but i was advised to start a new thread as the original question …

I have a 2d matrix with dimension (3, n) called A, I want to calculate the normalization and cross product of two arrays (b,z) (see the code please) for each column (for the first column, then the second one and so on).
the function that I created to find the ...

Write a C program that should create a 10 element array of random integers (0 to 9). The program should total all of the numbers in the odd positions of the array and compare them with the total of the numbers in the even positions of the array and indicate ...