Government inspector to decide on future of Grade II listed