Jacob Rees-Mogg said MPs may have to meet virtually (Image: Getty)
In a statement it said the temporary local relocation would be the "most secure, cost-effective and practical solution", with the plan being submitted to Parliament for final consideration.
It said that the review found that restoring the historic building while all parliamentarians remained on site would "cost billions of pounds more and take decades longer" than the temporary move.
But Mr Rees-Mogg told the Commons: "The proposal for Richmond House and for the Queen Elizabeth Centre was there would be about £1.5 billion of expenditure on temporary chambers. This can't have been a sensible thing to do, even in less straitened financial times. In current circumstances, it seems to me to be for the birds.