Kill everything and clear out the spawn points in the forest surrounding the Elder's location. This greatly reduces the number of enemies who interfere.