When Horatio Sharpe, the colonial governor of Maryland, started work on Whitehall in the 18th century, the story goes that he hoped the palatial villa on the Chesapeake Bay would win his beloved's heart. Alas, the young woman fell for Sharpe's friend and personal secretary instead.