I've hit that limit myself in a few place at varying depths, I didn't build a whole channel/canal, but I dug into land to dig out a boat dock... (used flying cheat to get into the water) got everything pretty well flat and deep enough except one area just wouldn't go down any further. Then I hit it again in the same area, but several meters higher up and back a bit, further away from the sea, the land in that area just won't flatten and I can't dig it down any further.

Basically as Mr. B described above.

I hope they fix/expand that at some point. I'd like to see either some more advanced terrain tools that allow you to do more and/or be more precise, or the terrain system in general improved so we don't hit "undiggable" dirt which, by all reasoning, should be very diggable.