You can't simply run a verb into は like that is the simple answer.The part before は is generally speaking a noun phrase. 'と' is, of course, a particle, but in this kind of case, it's essentially short hand for 'というのは' which would be a noun phrase followed by は.

Or, conversely, you can look at it as the phrase '全くないと言えないのである' with an extra 'は' inserted for 'emphasis' or 'contrast'. The phrase without the は is still valid, after all, but the phrase without the と isn't (unless you rephrase the sentence substantially.)