I killed the elder with no issues. Got a thick fog blow through the meadows and got some draugr in the fog, it was at night too. Only invasion is the trolls now which destroy my stairs but easy to take care of.