Billy Ivy was born in Grady, Arkansas to Pete and Maggie Ivy. They moved out to California when he was in second grade and ended up settling in Arcata. Billy attended Stewart School and graduated from Arcata High School in 1957. He worked at the Minor Theater during high school.