If the numbers in the matrix are known to be approximate, you have a difficulty if the determinant of the matrix seems to be zero (or very close to zero).

Very common misunderstanding! It is not a problem if the determinant is close to 0. It is the condition number that matters, not the determinant. Contrary to expectation, a small determinant does not indicate near-invertibility, nor it is a problem.

The above matrix, where the first two rows are similar to the corresponding rows in the matrix supplied by akerkarprash, but the third row is the difference of the first two rows, so that the determinant of the matrix is zero, doesn't have an inverse. Nevertheless, the website mentioned above calculates an inverse for it (as does wolframalpha). Of course the condition number is infinite in this case.

If I change the final decimal place of one of the entries in this matrix, the determinant of the matrix is very small and the above website calculates its inverse inaccurately, but the condition number is large.

For a matrix of this type, you are effectively asserting that its determinant can be very close to zero without its condition number being very large. Can you give an example of this?