Born March 30, 1901, Buford Alvin Townsend was a son of Reverend Wilson E. Townsend and Margaret M. Dixon. His father died at age 30 when Buford was five months old, leaving his 25-year-old mother to raise him and his two older brothers. They resided on a farm in the East Fork community and he received his early education in one-room rural schools, before he attended high school at Smoky Mountain Academy.
Townsend always worked to pay his expenses while attending high school. He obtained a teaching certificate and taught his first class when he was 17. Later, Townsend enrolled in two-year accredited college courses at Murphy College. While attending Murphy College, he taught Latin and math to help cover some of his expenses.