You collide with the func_ladder entity
I never really understood this. You collide with it yet it's invisible. It would make more sense to me if:
1) If solid,
but visible, you could make the ladder-textured brush a func_ladder and be done with it.
2) Invisible and not solid, it could be like an area trigger where if you're in it, you climb up any vertical surfaces, like the ladder-textured brush you're standing in front of.
When I was just getting started in the world of mapping, I first assumed (1). Noticing it was invisible after compiling, I assumed (2), and immediately imagined all sorts of dynamic ladder contraptions involving func_doors moving into the right place.
Turns out it was neither. It would have been awesome to have movable ladders, though.