Originally posted by Mikakunin:

you build castle after you unlock stonecutter which is around 3rd boss

You need only the second Boss to get the key for dungeons in the sump. There you get iron, this is need for the stone cutter.