prashanthch has asked for the
wisdom of the Perl Monks concerning the following question:

Hi,

I am trying to read data stored in a MySQL table, the column is defined as binary (8). Data can be double or uint64. To convert this data to double, I am using unpack ("d", $data) and for uint64, I am using unpack ("Q", $data).

With uint64, I appear to be getting right results for smaller numbers (3, 40, etc) but not for larger numbers. The other problem is that with double, the results do not appear to match.

Am I doing this right or should I be looking at some other way to convert other than unpack?

How much do you know about (and do you have access to) the process that puts these kinds of values into the database?

Also, I'm curious: are you saying that there is a single "binary(8)" column that is being used to store doubles in some rows and uint64 in other rows? If so, is there some other column in each row that tells you which data type is being stored there?

Since you seem to have some other way of knowing what "unpacked" value you should get for a given row, maybe it would help to look at the "raw" bits for that known field value (e.g. as a 16-digit hex numeric string). For uint64 values, that should clear up any problem involving endianness. (Working out the bit fields for doubles will be a little trickier.)

Unfortunately, I do not have access to the underlying routines that insert data into the database, I tried to get access but was denied, I am still trying

yes, there is a single binary(8) column that is used to store doubles for some rows and uint64 for others, another table in the database lists the type of data, I look up the type and run unpack appropriately

Taking a different look at those values might be instructive, given that they should have some sort of binary relation when treated as uint64 values. Here's a one-liner that will read the integer values typed (actually, pasted) into STDIN, and spit out the corresponding bit sequence for the given value:

Who could have guessed that the value actually returned by your unpack usage was identical to the value expected, except that it's missing the 3 lowest bits! What could be going on in your script (or in its interaction with the database) that might be causing this?

The double values involve some extra work (and I'm not sure I'm doing this the "right" way to suit your purposes...)

The first line of 64 bits is the "expected" value, the second is the value returned by your unpack, and the third has a "1" wherever the previous two don't match. I'm not sure how informative that is, actually, but it's curious to see the amount (and position) of agreement between the expected and returned values.

Still, as I said, there's a serious chance that this particular pursuit of the bit patterns in doubles is misguided - e.g. I may be using a different notion of "double" from the one being used to populate those binary fields.

UPDATE: Regarding the doubles, please note that adding an extra decimal place of precision to either (or both) the expected and returned values that you quoted will affect the position and amount of (dis)agreement between the two corresponding bit strings. You should look at the actual bit string as stored in the database, and see how many significant digits it takes to render it accurately as a decimal number (assuming you know how to interpret the binary(8) string correctly, of course).

ANOTHER UPDATE: For that matter, removing significant digits has an effect as well. Just for grins, I rounded off the "returned" value to 64852.468504 (removing the 3 least significant digits), and with that, the three binary strings (expected, returned, diff) came out like this: