If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

Re: Read binary file with line delimeter

convh is used to convert a hex char to decimnal. A hex number is char '0' to '9' and 'A' to 'F'. In ASCII there are 7 other characters between '9' and 'A' hence the 7 0s in the middle of the array. The first char is '0' so subtracting '0' from the char gives an index into convh that returns the decimal number. Using an array like this removes having to have an if statement for '0' to '9' and for 'A' to 'F'.

aindex is for the mapping. Valid values for '9X' are 0 - 4, 6, 7, 10 (A), 11(B). Again subtracing '0' from the 'X' code gives an index into this array. The value returned is the element position in sblk (starting at 0) - so '92' maps to 3. Where X isn't defined I used 9 to map to last position of sblk (which has 10 elements) in case one of these values happens to occur in some file.

All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.

Re: Read binary file with line delimeter

Thank you. I haven't tested yet, only would like to understand better.

Well, this goes back by learning to use the debugger and stepping through the program to see what each line does.

For example, what 2nd y 3rd array you define are for? And why in 2nd array you define some elements with zeros
And why in 3rd array you define some elements with 9's?

Code:

sblk[aindx[styp - '0']] = sub;

Those arrays are used in this line of code. It should be obvious that the inner array aindx returns an index that will be used by the sblk array. The " - '0'" part is the 'C' way of turning an ASCII digit into a number.

If you knew nothing else of the program, then you take this limited knowledge and run the program under the debugger to see what index is being returned and how doing so works.

Re: Read binary file with line delimeter

Hello 2kaud,

Thank you, is very close now the output, only I found that in attached file (bin1MB.txt) some blocks, beginning in block number 9128 (0023A8 in hex) is printing not the correct substrings, because in this block there is more than one "05" and maybe is because of that the confusion.

There is 0590.. (in red), then another "05" in blue and the correct 05 (in green) that is previous to 910f in this case.

Re: Read binary file with line delimeter

Currently, the search for subblocks starts straight after the first 05 following an ff79. It was assumed in the absence of information to the contrary that there was only one 05 block following the ff79. This new condition regarding 05 will be tricky as it's not a straight search. I'll have a look sometime and see what can be done.

All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.

Re: Read binary file with line delimeter

Thank you for your help. I've tested the last main function and it seems to print the correct output and is handling the issue mentioned in previous post. Besides that it seems to be very fast.

I've tested with a 20 MB and works fine, but with a 2GB file I get "segmentation fault" and stops around the block 425,000.

I saw the segment of the binary when stops, I saw 2 blocks before and they seems to be normal blocks, the only what I see is that the block number is not consecutive between the last 2 blocks and the last within the binary, for son reason. I'm not sure if the fact that 2 consecutive blocks don't have 2 consecutive block number can cause this issue.

Re: Read binary file with line delimeter

Before the latest code change, did it process all the blocks (with some incorrect output) for the 2GB file? I suspect there are some non-sub blocks that seem to look initially like sub-blocks but aren't. I've added some more tests to try to catch this. If the code below still gives the segmentation fault, can you extract a few records from the 20Gb file before and after the rouge block and zip and attach so I can investigate further. The more tests I have to add though, the slower and more complex the code becomes.

Re: Read binary file with line delimeter

Hello 2kaud,

With the previous code before last changes I didn't detected incorrect output for 2GB file.

Regarding the issue of segmentation fault I get, is as follow.
I have 2 types of files and I get segmentation error in file type2:
Type1: With SEPAR=FF77, SBLOCK=FF79, and substrings of the form "05 + 9X + .. + 940E + 14bytes"
Type2: With SEPAR=FF32, SBLOCK=FF34, and substrings of the form "03 + 8X + .. + 840E + 14bytes"

Both kind of files follows the same rules, only have differences mentioned above, so, I thougth if
it works for file of type1, changing only the SEPAR to FF32, SBLOCK to FF34 and intead "05" use "03" and
instead "9X" use 8X would work. Actually is working for almost all cases, but for some reason, for
the block 425987 (068003 in hex) in the attached file (Bin1KB.txt), I get segmentation fault and I don't see
much difference between this block an others. Only what I see is that has the SBLOCK string (FF34) but
don't exist the substring "03 + 8X + .. + 840E + 14bytes".

Why "smlblk" is initialized with "68"?

I was trying to see how "smlblk" works with "SBLOCK" in order to fix the issue with files of type2, but I
haven't understand yet, maybe in that part could be the solution.

PS1: The attached sample file is of type2 and only contains 6 blocks.
PS2: When I run the code ove Bin1KB.txt from IDE I don't get error, but block 425987 is not printed, I get error
when I run the built program.exe in command line, like this "program Bin1KB.txt"

Re: Read binary file with line delimeter

I've added a couple of defines at the beginning so that you can more easily select the file format you are dealing with. Just comment in/out the define you don't want. Another way would be to accept a file name and type option from the command line. If you can find a way of identifying the type of the file from the header than the program could auto detect the file type.

The code below parses bin1kb without any runtime errors.

However, I think the time is fast approaching when you need to take ownership of this program and understand it so that you can maintain it as required in the future. I'm not in a position where I can keep maintaining this program. This needs to become your responsibility. With a small file like bin1kb I suggest that you walk through the code on paper and use the debugger to see how it works against looking at a hex display of the data at the same time (if you don't know the debugger very well now is a good time to become familiar with it as using it is an essential skill). The big problem with parsing this data is to decide whether something that looks valid at the start is actually valid or not. The code is fairly simple and is the type of program I would be handing to a junior/trainee c++ programmer to maintain. Have a close look at it and try to understand. If there's anything you can't get just ask.

Re: Read binary file with line delimeter

Originally Posted by 2kaud

However, I think the time is fast approaching when you need to take ownership of this program and understand it so that you can maintain it as required in the future. I'm not in a position where I can keep maintaining this program. This needs to become your responsibility.

Hello 2kaud,

Many thanks for all the help in this case. Of course I need to understand it to be able to do some change, I don't want to
bother you anymore, too much help so far

I'll see how to use the debugger to see each part of the code, if you want to comment a little bit the code to give a better idea would be great, if not I undertand and is fine

Last question: Which compiler and IDE are you using? I tested Bin1KB.txt with your last code using Code Blocks with GNU GCC compiler and I continue receiving segmentation error. With Visual Studio 2012 the compilation finishes without errors, but when I run the program appears

Re: Read binary file with line delimeter

Originally Posted by Philidor

Last question: Which compiler and IDE are you using? I tested Bin1KB.txt with your last code using Code Blocks with GNU GCC compiler and I continue receiving segmentation error. With Visual Studio 2012 the compilation finishes without errors, but when I run the program appears

The only change I did to your last code is that I changed _itoa() with _itoa_s() as you suggested me in post #136, because without that change I get error in compilation in Visual Studio.

Thanks so much again for such help!

The assertion is telling you what the problem is. If you ran the program under the debugger, the assert() function tests to see if the expression is true. If it isn't true, the program stops. You then use the debugger to see what the call stack is that led to the function call, what the values of the variables are, etc.

Code:

length < sizeInTChars

The code stopped because that expression is false, when the program logic asserts that it must evaluate to true before proceeding.

* The Perfect Platform for Game Developers: Android
Developing rich, high performance Android games from the ground up is a daunting task. Intel has provided Android developers with a number of tools that can be leveraged by Android game developers.

* The Best Reasons to Target Windows 8
Learn some of the best reasons why you should seriously consider bringing your Android mobile development expertise to bear on the Windows 8 platform.