note that the raids will focus on the player character and not directly on the base
if you stick around in a house during a raid the mobs will start destroying the house in order to get to you

so, prepare proper defense or step outside and do it face to face or kite them around until the raid ends