A man charged with stalking and harassment after his arrest near singer Taylor Swift's Manhattan townhouse had been spotted there dozens of times in the past two months and was repeatedly asked to leave, according to court documents released Wednesday.