Originally posted by jonnin:

its a craft at location menu, under furniture, like a chest.
protip ... there are a ton of free lanterns in the world, making a bunch is a waste.

Yeah, unless you have a boatload of bronze you're not doing anything with, you can loot like 15 from a good mistlands with enough dvergr excavations or towers.