"There's no doubt it's a beautiful building, but it's the wrong building in the wrong spot," said one Sunny Isles Beach resident.