Well, I figured out what the problem was. I had created the cave with GenSurf, using a bitmap, and the "ceiling" option. When I created it, I was presented with what looked like the top of a cave, but when I tried to add a bottom to it and compile it, I kept getting brush outside of world errors, so I thought that my cave ceiling, was actually a hill.
Anyway, I eventually did a search on the brush outside of world error and figured out that the real problem was that I had the map too close to the edge of the map limits.
There were openings on the sides of my cave as well where the ceiling didn't quite reach the ground, those I tried to cover up by creating rectangular brushes and pasting them over top of the holes, I was actually trying to create a flat brush to do the job. But that looked sloppy to me, then I had an inspiration and deleted the bottom and all my sloppy patches, and instead created a box that surrounded the cave, giving it a neat bottom and sides, with no leaks.
I hate that feeling when you are such a nOOb that you don't even know enough to ask the right question to get help.
Anyway, to make a long story a bit longer. I added my little player spawn point, and a light, and compiled the map, after frigging around with textures for a bit, and hey presto, it worked! I was able to walk around in my first little map - a moment I will surely never forget.