The key precedent is Yick Wo v. Hopkins, a foundational case for constitutional law as the first Supreme Court case to apply the equal protection clause.
Perhaps the most grotesque aspect of the affirmative action opinion lies in its effort to drain the moral and legal force of Brown v. Board of Education.