My group was also at the point that pure vanilla became too easy. Then we tried lightly modded server (just some better foods and enemies up to 3 stars). It was fun but we had some troubles with tweaking mods to our likes. It was still too easy. Enemies still had the same attack patterns and we still used same tactics in same situations. Nothing really changed except the amount of hits we needed to kill them.
Then we decided to go back to vanilla but with NOMAP. Now, this is a proper challenge. Everything is still the same but we needed to change our approach to everything. And there is a lot extra things we have to think about.

If you want to scale the difficulty without changing anything, try using only light armors. Try playing without upgrades to your weapons. Or maybe a play through without a cauldron, so basically you eat only berries and meat cooked on cooking racks. I even played without a bow on one world. There is so much combinations of rules you can set up for yourself.