// I think slavery was a horrible concept, but it was of its time, it happened, and me being 'sorry' will not make it un-happen... //
There are a few points here.
Firstly, it's not really about the individual being "sorry", so much as the State, which tends to claim continuity with its history, but therefore ought to accept the consequence of that, including responsibility for all of the State's previous actions, not just the positive ones that can be celebrated. You may well argue that the "State" shouldn't have to apologise either, perhaps on the grounds that the continuity is more of a technicality, but I mention it mainly to stress that individuals apologising is not really and never has been the point.
Secondly, the "of its time" is overly simplistic: it ignores the opposition even then. For example, Elizabeth I is said to have written that, "if any Africans should be carried away without their free consent, it would be detestable, and call down the vengeance of Heaven upon the undertakers." It would be odd indeed to claim that she was the only person in the Realm at the time who condemned it, and while in the end her position either changed later, or was tempered by economic aspects, the fact remains that she said this and understood almost before it began that the Slave Trade was wrong. Many others, in a less powerful position, might well have chosen to keep their heads down, or turn a blind eye, since slavery being vital to the economy for a long time meant opposition to it potentially ruinous personally.
Thirdly, whatever the British thought of it, pretty sure that the Slaves saw that what was happening to them was wrong. Why is their morality less relevant? Some people might at this point mention that the Slave Trade was greatly benefited by local African tribe leaders, selling off rival tribe members, and they are (sadly) correct, but these too are people in power rather than the victims, whose perspective remains ignored even with this extra information.
But the main points are that (1) Slavery still exists, so maybe "was" is premature; and (2) if Slavery (in this sense) were acceptable once then it can be seen as acceptable again in future. We must never allow that: therefore, we must continually condemn it.