The raid sticks around for a certain amount of time (a couple minutes), and only goes down when you're in the area. Try to fight if you can, but simply running in circles until the enemy grows bored is an option, though the last enemies in the area will still chase until they've been dealt with.