Slavery's been pretty much universal, and in fact there are more slaves now than in 'slavery days'. The unique things about the Atlantic trade were its brutality, the numbers involved, the amount of money made, and the way Africa was systematically destabilised in order to maintain the flow of prisoners. In most situations, slaves had legal rights, and at least some chance of getting back home. In many, they could achieve high office. There was none of that in plantation slavery.
It was the Jamaican planters who originally turned it into a black-white thing. By far the greatest number of slaves were black, and when traditional justifications for it stopped working as the anti-slavery movement grew, they turned to race. A guy called Edward Long published his History of Jamaica in 1774, in which he argued that Africans were closer to apes than they were to Europeans. Unfortunately I can't lay my hands on any direct quotes right now, they'd shock you. Other people followed his lead, and eventually his ideas evolved into the idea of white superiority and 'the White Man's burden' which were used to justify colonialism.