Business suffers as once vibrant sugar millers go silent on