Not really sure with greydwarves, but I did have a Neck sneak in my base through a hole under the wall. Had to fix up the landscape a bit to make it all fit better.

Also workbenches seem to stop nearby random spawns, so you might have to hide a few around your base to prevent spawns inside the walls. Just make sure the radius of the workbenches covers the area.