You cant dig a "cave" into the side of a hill or anything, and likely that won't change as they are already about halfway done with the game with the mechanics already in-place, however, There are ways to live underground if one uses some creativity and imagination.

One thing I often do is create a workshop outpost (for processing bronze) right at the source, rather than transporting metals and ores across the map, to some distant base. To make a fairly safe place for this I often find an area with several copper nodes fairly close together, and use one of the copper nodes as an underground house/workshop by digging straight down, along the edge of the copper node then digging out a hollow space inside, leaving the top "cap" of the copper node intact to use as a durable "roof" cover...

https://steamcommunity.com/sharedfiles/filedetails/?id=2858958865

https://steamcommunity.com/sharedfiles/filedetails/?id=2858960285

https://steamcommunity.com/sharedfiles/filedetails/?id=2858959032

It's not a HUGE amount of space, but theres plenty of room to place the essentials for processing bronze, cooking facilities, portal, bed etc. and safe enough that I have never had any mobs bother me down there.

Another, larger "underground" area I use a LOT is the space that can be dug out underneath the Elder altar... If you find one that is suitably higher than the water table, you can end up with a massive amount of space to build completely safe underground bases...

I generally dig straight down along one edge of the altar then hollow out the entire space underneath, (in this case i put stone walls all around the edges, making a completely safe place for a home.

https://steamcommunity.com/sharedfiles/filedetails/?id=2855621495

After digging it out, I thoroughly enjoy experimenting and creating beautiful homes under there.

https://steamcommunity.com/sharedfiles/filedetails/?id=2434836833

https://steamcommunity.com/sharedfiles/filedetails/?id=2858966733

https://steamcommunity.com/sharedfiles/filedetails/?id=2858869195

https://steamcommunity.com/sharedfiles/filedetails/?id=2858867908

https://steamcommunity.com/sharedfiles/filedetails/?id=2858965569

https://steamcommunity.com/sharedfiles/filedetails/?id=2847464543

Truly the possibilities are endless.

PS. Everything shown is 100% vanilla (un-modded)