Saadia Group, Lord & Taylor’s new owner, is taking the once-sprawling retail empire, which had anchored a 660,000-square-foot building on Fifth Avenue, and packing it into three floors of a Midtown office.
The investment firm, which acquired retailers Lord & Taylor and New York &…