To be fair, ladders are either broken or "needlessly complex" in pretty much all 3D games. It seems that snapping into the "climbing a ladder" position and changing the control scheme to one that won't fling you off the ladder into empty space "for no reason" is difficult... as is detecting when the player wants to actually use the ladder.

The best one I've ever seen was in Team Fortress Classic; I'm fairly certain it was based on proximity to the ladder, causing you to snap onto the ladder if you got close to it. If I recall correctly, the controls were adjusted so that moving "toward the ladder" was "up", and "away from the ladder" was "down", while requiring the user to press "jump" to let go of the ladder.

Some games that have ladders require the user to press the "use" button to "attach to the ladder", and climbing either "feels bad" or is so slow as to make it faster to go around the ladder and use whatever ramps or jumping puzzle gets the player there without having to interact with the ladder.

Other games that use proximity-based ladder sensing often get slammed by the playerbase for grabbing onto the ladder while running past it; when it comes to ladders that "feel good" to use, it really does seem to be hit-or-miss.

It should also be noted that my "best example I can remember from 40+ years of playing video games" still had the occasional weirdness when releasing from the ladder, occasionally (maybe one time in 20) flinging your character 20-30 feet in a random direction when you let go of it.

TL;DR: Vertical ladders in 3D games are apparently hard. The few games that implement them have wonky behavior, and or require an immersion-breaking "mode shift" to "ladder climbing mode".