Now, we are supposed to show that the ratio test is inconclusive for this series. But when I apply the ratio test, I get:

(n+1)^(1/2)/((1+(n+1)^6)*the original series and nothing seems to be cancelling out.

OK, first of all, you should be dividing (n+1)^(1/2)/((1+(n+1)^6) by the original term, [n^1/2]/[1+(n^6)]. Secondly, nothing will generally cancel: you are supposed to take the limit of the absolute value of this ratio as n approaches infinity. Group the "like factors" together to form ratios like [1+(n^6)]/[1+({n+1}^6)] and look at the infinite limit of those ratios. You will find that the limit of the product of these ratios you've formed gives you 1 (i.e., the Ratio Test is useless here).