To be fair, the only reason that the north outlawed slavery is because they didn't have much use for them. The land was much too rocky to do anything but subsistence farming and small scale commercial farming. The south was suited for large scale farming, and that needed a lot of man power.

The north was either industry, shipping, or whaling. They had a lot of use for slavery, and sure, some of them were okay with it or used slaves, but for the most part, it was a moral issue for the North.