Norton was born enslaved about 1840 in Williamsburg of mixed-race ancestry. He was a younger brother by about a decade of F. S. Norton, who was also a member of the House of Delegates (1869–1871), and an elder brother of Daniel M. Norton, who was a member of the Convention of 1867–1868 and of the Senate of Virginia (1871–1873, 1877–1887). They may have been sons of an enslaved woman and her owner. The identities of their parents are not known for certain, and Robert Norton provided different names for his mother on each of his two marriage records: Richard and Elizabeth without surnames in one instance, and Charlotte E. Norton in another.
Norton was born enslaved early in the 1840s in Williamsburg. His brother F. S. Norton, a member of the House of Delegates (1869–1871), was older by more than a decade. He was close in age to Robert Norton, another brother who was a member of the House of Delegates (1869–1874, 1876–1883), and they were reportedly the sons of an enslaved woman and her owner. The identities of their parents are not known for certain, but Robert Norton provided different names for his mother on each of his two marriage records: Richard and Elizabeth without surnames in one instance, and Charlotte E. Norton in another. About the middle of the 1850s, Norton and his brother Robert escaped from slavery in Gloucester County to freedom in Troy, New York. He received medical training from a doctor there and on January 23, 1860, married Edmonie, last name unknown, in Philadelphia. Two days later, she traveled to Virginia, where he refused to go, fearing re-enslavement. They did not often live together befo