The evacuations started in the early hours of Monday and are expected to continue.
Schools in the area are also being evacuated.
Warm clothes, children and pets were bundled into cars as confused and scared Vredehoek residents gathered at the Derry Street Spar in the early hours of Monday.
They were the first residents to be evacuated from Peppertree Road and surrounds, as firefighters tried to douse a runaway fire that started on Sunday morning.
The fire has already destroyed the Rhodes Memorial restaurant, the iconic Mostert Mill and historic UCT buildings.
More than 250 firefighters have been battling the blaze for over 24 hours, while four firefighters have been injured, said the City of Cape Town s Fire and Rescue Service spokesperson, Jermaine Carelse.