Raids don't attack your base... they target you.
Don't spend too much time at base... get out, explore, keep moving.