11 states wanted to keep slavery
22 states did not.
So it seems to me there were a heck of a lot more white people wanting to give blacks their freedom than keep them in slavery. A more accurate question would be why were most white people against slavery.