We should welcome ex-PMs back to cabinet, but Cameron won't