Thus, out of all the factors, civil wars and usurpations arguably were the biggest downfall of the empire, along with the Fourth Crusade, which was the nail in the coffin of the persistent Roman Empire. Thus, out of all the factors, civil wars and usurpations arguably were the biggest downfall of the empire, along with the Fourth Crusade, which was the nail in the coffin of the persistent Roman Empire.