Raids do not attack your base... they attack you -_-
If you aren't nearby or unreachable, then they will target your buildings.
Also, raids are already weighted to appear the longer you remain close to any of your building/property.