Sword of Light is weapon damage, not all damage. It'll improve the damage of Crusader Strike, Templar's Verdict, Hammer of the Righteous, Divine Storm, Seals (but not Censure), and auto-attacks. While still quite a few attacks, that's not quite "all" damage--notably, it won't affect Exorcism, Judgment, Hammer of Wrath, Censure, or Harsh Word if you use that. :P