I trained a BERT model (Devlin et al, 2019) from scratch on my desktop PC (which has a Nvidia 3060 Ti 8GB GPU). The model architecture, tokenizer, and trainer all came from Hugging Face libraries, and my contribution was mainly setting up the code, setting up the data (~20GB uncompressed text), and leaving my computer running. (And making sure it was working correctly, with good GPU utilization.)
Victoria Graf, Amélie Lemay, Arya Maheshwari and Reha Mathur have been awarded Goldwater Scholarships, an annual award for outstanding undergraduates interested in careers in mathematics, the natural sciences and engineering.