James Thomas Sammons Taylor was born on January 14, 1840, in Berryville. He was the son of Fairfax Taylor, an African American shoemaker who had purchased his freedom, and his first wife Ellen Sammons Taylor. By 1850 the family had moved to Charlottesville, where his father became a prominent leader in the black community. He reportedly hired a private tutor for his son and also taught him the cobbler’s trade. After the U.S. War Department established the Bureau of Colored Troops, James Taylor enlisted in the USCT’s 2nd Infantry Regiment on August 24, 1863, in Washington, D.C. Assigned to Company E, he was named acting commissary sergeant and was promoted to the post on November 10, 1863, with the rank to date from September 1.